1
00:00:02,360 --> 00:00:02,960
Hi guys.

2
00:00:02,960 --> 00:00:03,880
Welcome to this session.

3
00:00:03,880 --> 00:00:08,120
So in this session we'll talk about what are tokens and token limits.

4
00:00:08,520 --> 00:00:13,920
So tokens are basically these are LM processes the text using tokens.

5
00:00:14,080 --> 00:00:21,200
So the large language models like ChatGPT which we are using, they process the text which we see using

6
00:00:21,200 --> 00:00:25,720
these tokens which are common sequences of characters found in a text.

7
00:00:25,760 --> 00:00:31,600
These models learn to understand relationships between tokens and then predicting the next one in a

8
00:00:31,600 --> 00:00:32,400
sequence.

9
00:00:32,440 --> 00:00:36,760
Like in this example, you can see the sentence which we're given is.

10
00:00:36,760 --> 00:00:43,560
Today is a beautiful sunny day that has utilized seven tokens and has 35 characters.

11
00:00:43,760 --> 00:00:49,760
So seven tokens is each word which we get to see gets to use one token.

12
00:00:49,880 --> 00:00:54,200
And then also the period at the end is given a specific token.

13
00:00:54,600 --> 00:01:00,800
Now you can count the total number of tokens in your piece of text by going to the open AI platform's

14
00:01:00,800 --> 00:01:06,480
tokenizer section, and you can see for yourself how many it might tokenize.

15
00:01:06,840 --> 00:01:13,950
So now looking at the token limits, this is going to be the maximum number of tokens that a model or

16
00:01:13,950 --> 00:01:16,270
an API actually processes.

17
00:01:16,430 --> 00:01:22,950
Now this takes into consideration both the input tokens which is our prompts, and the output tokens,

18
00:01:22,950 --> 00:01:25,350
which is the results which we get from the tool.

19
00:01:25,670 --> 00:01:33,470
Now, for example, GPT four has a context window of 128,000 tokens it can provide now, which is almost

20
00:01:33,470 --> 00:01:36,630
equivalent to 500 book pages.

21
00:01:36,910 --> 00:01:43,790
Now we use the max token parameters in the API to limit the output tokens.

22
00:01:43,790 --> 00:01:49,950
So which is basically done for the purpose of making sure that the output which we are giving are concise

23
00:01:50,110 --> 00:01:51,030
and readable.

24
00:01:51,510 --> 00:01:54,310
Now we also want to control the API cost.

25
00:01:54,350 --> 00:02:01,230
We want to save computational power and also ensure that the results are below a certain length.

26
00:02:01,510 --> 00:02:06,270
So these are the this is the tokens which is being used at the back end.

27
00:02:06,310 --> 00:02:11,230
Every time a user is prompting the AI tool and it gets the responses.