संपर्क करें

Understand LLMs from Zero#7

What Are LLM Parameters? A Model Is One Giant Function

A model is a function: numbers go in, numbers come out. The numbers that decide how it behaves are its parameters. On this page you turn the knobs (the parameters) yourself and watch the same box start doing a different job. At the end you see the results of an experiment that swaps the parameters of a real LLM.

  • POINT 1A function is a box: a number goes in and a number comes out by a fixed rule. The same input always gives the same output.
  • POINT 2The numbers that decide how a function behaves are its parameters. Change their values and the same box does a different job.
  • POINT 3An LLM is one giant function defined by about 3.9 billion parameters. Their values and their places decide everything.

TRY 1

Turn the knobs and change the job of the box

The rule of this box is “output = a × input + b”. The two numbers a and b are its knobs (its parameters). Keep the formula as it is, change only the knob values, and make the box do three different jobs. You can drag a knob, use its slider, or press − and +.

output = 1 × input + 0

1
0
Choose the job for the box:
Goal: set the knobs so the box does “double it and add 1”
inputtarget outputoutput now
131
252
373
5115
102110

Turn the knobs until the output matches the target. 0 of 5 rows match so far.

The bars are the current outputs for the inputs in the table. A bar turns green when it matches its target. Everything is computed in your browser.

TRY 2

More knobs can describe more complicated relations

Make the light-blue line follow the orange dots. With 2 knobs you can only draw a straight line. With 4, and then 8, the line can follow a winding row of dots. Doing it by hand is hard, so there is a button that fits the knobs automatically.

Choose the number of knobs:distance from the dots: 0.593
0
0

With 2 knobs the best possible distance is 0.297. Turn the knobs, or press “Fit automatically”.

The line is k1 + k2·x + k3·x² + … “Fit automatically” computes the knob values with the smallest distance (least squares). For a real LLM, the work that corresponds to this button is called training.

TRY 3

Swap the parameters of a real LLM

A real model (Gemma 3 4B) has about 3.9 billion knobs. What happens to its output if not a single value is changed and only their places are swapped? Turn the knob to raise the share that is moved. Every candidate and every continuation was measured by actually running the model.

0%

What the model wrote nextThe capital of France is Paris. ⏎ Paris is located in the north-central part of France

This is the original model. “Paris” is first with 85%, and the continuation is right. Turn the knob and move some parameters.

Choose the input sentence:

A few of the real parameters (there are 3,880,099,328 in all; the file is about 2.49 GB). All of them are small numbers close to zero.

  1. +0.01539
  2. −0.00377
  3. +0.00006
  4. −0.00377
  5. +0.01156
  6. −0.00760
  7. +0.00773
  8. −0.01143
  9. −0.00377
  10. −0.01143
  11. −0.02293
  12. +0.01156
  13. −0.01143
  14. +0.00389
  15. +0.00006
  16. +0.01922
  17. +0.01922
  18. +0.01539
  19. +0.01156
  20. −0.00377
  21. +0.00773
  22. −0.03826
  23. +0.01539
  24. −0.03826

The parameters are moved in blocks of 256 neighbours, to another place inside the same component (the result of one random draw). At the same share, how it breaks depends on which blocks are hit. That is why only one of the two sentences is broken at 0.3%.

CHECK

Check yourself

Three questions about what we learned today.

Q1What is a parameter?

Q2You feed the same sentence in twice. What about the next-token probabilities?

Q3All parameters keep their values but are moved to other places. What happens?

The candidates, probabilities and continuations in “Swap the parameters of a real LLM”, and the parameter values, were all measured on a real language model (Gemma 3 4B, 3,880,099,328 parameters). The results were computed in advance, so nothing is sent to a server. The knob calculations are done by your browser on the spot.