Anfragen

Understand LLMs from Zero#25

How Does an LLM Know Word Order? The Position Problem

“The dog bit the man” and “The man bit the dog”: the same words, only the order differs. Yet if you add up the embedding rows of their tokens, the two sentences become exactly the same 2,560 numbers. Addition does not remember the order. Shuffle the words yourself and see.

  • POINT 1The same token gets the same row of the embedding table, wherever it stands in the sentence. Nothing in the row says which place it is in.
  • POINT 2Add the rows together and the order is gone. Sentences made of the same tokens become one and the same vector, whatever their order.
  • POINT 3But a real LLM does change its answer with the word order, because position information comes in somewhere. How that works is the subject of episodes 46 and 47.

TRY 1

Shuffle the words: the sum does not change

Press two of the coloured words one after the other, and they trade places. In the 3D view the token blocks change places, and the bars behind them (the first 64 numbers of the sum) are added up again in the new order. The white lines mark the sum of the original sentence.

Sentence

Press two words to swap them (grey parts stay where they are)

The

Now:The dog bit the man

Sum of the original sentence (first 128 numbers; orange = positive, blue = negative)

Sum of the sentence now

Of the 2,560 numbers of the sum, 0 differ from the original sentence (largest difference: 0)

Gemma 3's answer: after “The dog bit the man. The one that got hurt was the”, the next token is “man” with probability 99%, “dog” with 0.6%. The most likely one is “␣man” (99%).

The sum is computed in your browser: the real rows (2,560 numbers each) are added in the current order. Whatever the order, not one number differs. Gemma's answers, on the other hand, were measured beforehand for every order, and they change a lot. Gemma received the tokens in exactly this order (including orders that are not proper sentences).

TRY 2

Two sentences from the same tokens

Press the tokens below to build sentence X and sentence Y. The goal: two sentences in different orders whose sums are exactly the same. Also try replacing one token by another and see what happens.

Tokens (a press adds the token to the end of the sentence being edited)

Sentence X

Sentence Y

Done. The orders differ, and yet all 2,560 numbers of the sums are the same. From the summed vector, these two sentences cannot be told apart.

X and Y are the same bag of tokens. The tokens of X can be ordered in 120 ways, and every one of them gives the same sum.

The sums are equal exactly when the two sentences hold the same tokens, the same number of times each. The order leaves no trace in the sum.

TRY 3

How many orders are there?

n different words can be put in n × (n − 1) × … × 2 × 1 orders. With more words this number explodes. Addition stacks all of those orders onto one single point.

120

5 words can be ordered in 120 ways (3 digits).

“The dog bit the man” is 5 tokens. All 120 orders were computed: there was only one sum.

WordsOrders
11
22
36
424
5120
6720
840,320
103,628,800
12479,001,600
151,307,674,368,000
202,432,902,008,176,640,000

This is episode 23 again: squash it flat and you cannot undo it. When many sentences land on one point, that point cannot tell you which sentence it was.

TRY 4

All 24 orders: what the real Gemma answered

The four movable parts were put in all 24 possible orders, and Gemma 3 4B had to predict what comes next. A bag (a sum) would give the same answer 24 times.

Sentence

Every order was followed by the same “…. The one that got hurt was the”, and we measured the probability that the next token is “man”, and that it is “dog”.

The probability of “man” went from 1.7% to 99%, depending on the order.

OrdermandogMost likely
The dog bit the man 99% 0.6%␣man
The dog bit man the 93% 3.9%␣man
The dog the bit man 34% 62%␣dog
The dog the man bit 41% 22%␣man
The dog man bit the 6.6% 38%.
The dog man the bit 12% 25%␣dog
The bit dog the man 24% 25%␣dog
The bit dog man the 4.5% 13%␣bit
The bit the dog man 7.7% 31%␣dog
The bit the man dog 18% 18%␣one
The bit man dog the 3.4% 9.0%␣one
The bit man the dog 13% 28%␣dog
The the dog bit man 92% 6.9%␣man
The the dog man bit 7.2% 49%␣dog
The the bit dog man 6.0% 9.4%␣one
The the bit man dog 1.7% 5.4%␣one
The the man dog bit 52% 25%␣man
The the man bit dog 13% 82%␣dog
The man dog bit the 72% 20%␣man
The man dog the bit 8.6% 27%␣dog
The man bit dog the 33% 65%␣dog
The man bit the dog 15% 83%␣dog
The man the dog bit 83% 6.3%␣man
The man the bit dog 21% 13%␣one

The blue row is the original sentence, the orange row the one with the opposite meaning. With the same tokens in another order, Gemma gives another answer. So somewhere it is told which place each token is in. The embedding table at the entrance holds no position information. Where and how position comes in is the subject of episodes 46 and 47.

TRY 5

What the bag still tells; other sentences; the token split

What the summed vector remembers and what it does not, in measured numbers.

We searched all 262,144 rows of the embedding table for the ones closest to the sum of “The dog bit the man” (cosine similarity). Of the top 10, 5 are tokens of the sentence (green).

␣the0.68␣The0.64The0.62␣a0.54␣man0.52␣THE0.51the0.51␣dog0.50␣in0.45␣bit0.45

The sum of the original sentence against the sums of other sentences

SentenceDistanceNumbers that differ (of 2,560)
The man bit the dog00
The dog bit the boy1.212,557
The cat saw the bird2.242,560

A note on choosing examples: the classic headlines “Dog bites man” and “Man bites dog” do not work here. For Gemma, “Dog” at the start of a sentence and “ dog” after a space are different tokens, so the two headlines are not the same tokens in another order. Starting both sentences with “The” fixes that.

A curiosity: asked “The one who got bitten was the”, Gemma answered “man” for both “The dog bit the man” and “The man bit the dog” (96% and 93%). A man biting a dog is so unusual that common sense seems to win. With “got hurt”, the second sentence gives “dog” 83%. That is the wording used in this lesson.

Cosine similarity was the subject of episode 18; reading the contents back out of a sum, of episode 24.

CHECK

Check yourself

Three questions on what this episode showed.

Q1The token “ dog” stands second in one sentence and fifth in another. Which rows of the embedding table are looked up?

Q2Add up the rows of all tokens of “The dog bit the man”, and of “The man bit the dog”. The two sums are …

Q3Can the real Gemma tell apart two sentences that differ only in word order?

The sums (TRY 1 and 2) are computed in your browser; nothing is sent to a server. The rows are real values read from the embedding table of Gemma 3 4B (Q4_K_M) (gemma-3-4b-it-Q4_K_M.gguf); Gemma's answers were measured beforehand.

FINISH

Finished the episode?

COMMENTS

Comments & questions

Something unclear, something you want to know more about, or just your thoughts: write it here. We answer quickly, and other learners can reply too.

Loading…