How LLMs Generate Text: Autoregressive, One Token at a Time
An LLM does not write a text in one go. It predicts one token, adds it, and predicts again — over and over. Here you walk through that motion yourself, one step at a time.
- POINT 1A text is made by adding one token at a time (autoregression).
- POINT 2A token that has been written cannot be taken back, and it shapes everything after it.
- POINT 3Generation stops when the stop token is chosen.
TRY 1
The unit is a token, not a word
In episode 1 we said "one word at a time", but the unit an LLM really works with is the token. A word can be exactly one token, or it can be split into several. Look at the real pieces.
Once upon a time, there was a little girl.
- Once
- ·upon
- ·a
- ·time
- ,
- ·there
- ·was
- ·a
- ·little
- ·girl
- .
This sentence has 42 characters and 11 tokens. "·" stands for a space, "⏎" for a line break.
The number is the ID the model gives to that token. What the model actually receives is not letters but this sequence of numbers.
TRY 2
Add one at a time, until the answer is done
An answer to a question is also built by adding one token at a time. Each time you press the button, the AI gives probabilities to the candidates for the next token and adds the first one. The pink bar is the probability of the stop token — watch when it rises.
YouWhat is the capital of France? Answer in one short sentence.
AI
Press the button to predict the first token.
This model's stop token is <end_of_turn>. It is never shown on screen, but it is just one more token in the vocabulary.
TRY 3
Once written, there is no going back
An LLM cannot erase a token and rewrite it. If a token other than the first choice is picked, the rest of the text simply follows that token. Pick one yourself and see.
The capital of France is?
Choose the next token — the first choice or another one.
- ·Paris85%
- ·Lyon<0.1%
- ·London<0.1%
- ·a2.0%
TRY 4
The same beginning, a different text every time
If the token is drawn like a lottery ticket according to the probabilities, instead of always taking the first one, the same beginning gives different texts. Press the button and generate five times.
Once upon a time, there was a?
Let's generate from the same beginning, drawing by chance.
These are five results really generated with different random seeds. How the drawing works is the subject of episode 3.
CHECK
Quick check
Three questions about what you learned today.
Q1What is the unit in which an LLM builds a text?
Q2When does an LLM stop writing?
Q3What happens if a token other than the first choice is picked on the way?
NEXT #3
LLM Output Is a Probability Distribution: Next-Token Prediction
Next time: how is one candidate chosen? We look inside "probability" and the drawing of lots.
Every token and probability on this page is the output of a real language model (Gemma 3 4B). The results were computed in advance, so nothing is sent to a server.