Konzultáció

Understand LLMs from Zero#2

How LLMs Generate Text: Autoregressive, One Token at a Time

An LLM does not write a text in one go. It predicts one token, adds it, and predicts again — over and over. Here you walk through that motion yourself, one step at a time.

  • POINT 1A text is made by adding one token at a time (autoregression).
  • POINT 2A token that has been written cannot be taken back, and it shapes everything after it.
  • POINT 3Generation stops when the stop token is chosen.

TRY 1

The unit is a token, not a word

In episode 1 we said "one word at a time", but the unit an LLM really works with is the token. A word can be exactly one token, or it can be split into several. Look at the real pieces.

Once upon a time, there was a little girl.

  1. Once
  2. ·upon
  3. ·a
  4. ·time
  5. ,
  6. ·there
  7. ·was
  8. ·a
  9. ·little
  10. ·girl
  11. .

This sentence has 42 characters and 11 tokens. "·" stands for a space, "⏎" for a line break.

The number is the ID the model gives to that token. What the model actually receives is not letters but this sequence of numbers.

TRY 2

Add one at a time, until the answer is done

An answer to a question is also built by adding one token at a time. Each time you press the button, the AI gives probabilities to the candidates for the next token and adds the first one. The pink bar is the probability of the stop token — watch when it rises.

YouWhat is the capital of France? Answer in one short sentence.

AI

Press the button to predict the first token.

This model's stop token is <end_of_turn>. It is never shown on screen, but it is just one more token in the vocabulary.

TRY 3

Once written, there is no going back

An LLM cannot erase a token and rewrite it. If a token other than the first choice is picked, the rest of the text simply follows that token. Pick one yourself and see.

The capital of France is?

Choose the next token — the first choice or another one.

Choose the token to add
  • ·Paris85%
  • ·Lyon<0.1%
  • ·London<0.1%
  • ·a2.0%

TRY 4

The same beginning, a different text every time

If the token is drawn like a lottery ticket according to the probabilities, instead of always taking the first one, the same beginning gives different texts. Press the button and generate five times.

Once upon a time, there was a?

    Let's generate from the same beginning, drawing by chance.

    These are five results really generated with different random seeds. How the drawing works is the subject of episode 3.

    CHECK

    Quick check

    Three questions about what you learned today.

    Q1What is the unit in which an LLM builds a text?

    Q2When does an LLM stop writing?

    Q3What happens if a token other than the first choice is picked on the way?

    Every token and probability on this page is the output of a real language model (Gemma 3 4B). The results were computed in advance, so nothing is sent to a server.