A model does not read words. It reads tokens. A token is a small piece of text, sometimes a whole word, sometimes part of one. As a rule of thumb, 100 tokens is about 75 words.
Counted in Both Directions
Everything you send to the model is input: your question, your documents, the conversation so far. Everything it writes back is output. And output usually costs several times more than input.
A Worked Example
The video uses round numbers. You send a 10-page document and ask for a one-page summary. That is about 5,000 tokens in and 500 out. If input costs $3 per million tokens and output $15, the request costs about two cents.
Two cents sounds like nothing. But an agent working on a task may make hundreds of such calls, and it rereads its whole history each time. So the tokens, and the bill, grow much faster than the work seems to.
Input is ten times the output here, yet output costs several times more per token. Round numbers for illustration.
Why It Matters
This is why the token is the basic unit of the AI economy. Models are priced in tokens. Chips are measured in tokens per second. Data centres in tokens per watt.
Tokens also set a limit. A model can only hold a fixed number at once. That is its context window. Long documents and long conversations fill it up.
The Business Engineer framework maps where each part of the AI stack sits and who holds the leverage. These pills are short lessons from the same library.
Gennaro is the creator of FourWeekMBA, which reached about four million business people, comprising C-level executives, investors, analysts, product managers, and aspiring digital entrepreneurs in 2022 alone | He is also Director of Sales for a high-tech scaleup in the AI Industry | In 2012, Gennaro earned an International MBA with emphasis on Corporate Finance and Business Strategy.
Scroll to Top
Discover more from FourWeekMBA
Subscribe now to keep reading and get access to the full archive.
