Kontakta oss

LEARN

Understand LLMs from Zero

These pages go with the 100-episode video series. After each video, run the same mechanism yourself and type in your own words to see what happens.

Part 1The Big Picture

  1. #1What Is an LLM? Large Language Models Predict the Next WordOpen
  2. #2How LLMs Generate Text: Autoregressive, One Token at a TimeOpen
  3. #3LLM Output Is a Probability Distribution: Next-Token PredictionOpen
  4. #4Bigram Language Model: The Simplest Language Model, Built by CountingOpen
  5. #5The Limits of N-gram Language Models: Why LLMs Need Neural NetworksOpen
  6. #6How LLMs Turn Text into Numbers: Token IDs ExplainedOpen
  7. #7What Are LLM Parameters? A Model Is One Giant FunctionOpen
  8. #8The Full LLM Pipeline: Tokenization, Transformer, GenerationOpen

Part 2Tokenization

  1. #9What Is Tokenization? Characters vs Words vs Subword TokensOpen
  2. #10Byte Pair Encoding (BPE) Part 1: Merging the Most Frequent PairOpen
  3. #11BPE Part 2: Training a Tokenizer VocabularyOpen
  4. #12Tokenizer Encode and Decode: From Text to Token IDs and BackComing soon
  5. #13Why Token Counts Differ by Language: LLM Tokenization ComparedOpen
  6. #14Special Tokens and Chat Templates: How an LLM Reads a ConversationComing soon
  7. #15What Is a Context Window? How Much an LLM Can Read at OnceComing soon
  8. #16Why LLMs Fail at Spelling and Arithmetic: Tokenization PitfallsComing soon

Part 3Vectors and Embeddings

  1. #17What Is a Vector? How LLMs Represent Words as NumbersComing soon
  2. #18Dot Product and Cosine Similarity: How LLMs Measure MeaningComing soon
  3. #19One-Hot Vectors and Their Limits: Why LLMs Need EmbeddingsComing soon
  4. #20What Is an Embedding? The Token Embedding Table ExplainedComing soon
  5. #21Word Embedding Space: Why Similar Words Sit Close TogetherComing soon
  6. #22Word Vector Arithmetic: King − Man + Woman = Queen (word2vec)Coming soon
  7. #23What Is a Matrix? The Linear Transformations Behind Every LLMComing soon
  8. #24High Dimensions: Why LLM Embeddings Have Hundreds of DimensionsComing soon
  9. #25How Does an LLM Know Word Order? The Position ProblemComing soon

Part 4How Neural Networks Learn

  1. #26What Is a Neuron? Weights and Biases in a Neural NetworkComing soon
  2. #27Activation Functions (ReLU, GELU): Nonlinearity in Neural NetworksComing soon
  3. #28Multilayer Perceptron (MLP): Stacking Layers in a Neural NetworkComing soon
  4. #29The Softmax Function: Turning Logits into ProbabilitiesComing soon
  5. #30Loss Function and Cross-Entropy: Measuring an LLM's MistakesComing soon
  6. #31What Is a Gradient? The Direction of Neural Network LearningComing soon
  7. #32Gradient Descent by Hand: The Core Algorithm of Machine LearningComing soon
  8. #33Backpropagation Part 1: The Chain RuleComing soon
  9. #34Backpropagation Part 2: Computation Graphs and AutogradComing soon
  10. #35Mini-Batches and Learning Rate: Step Size in Neural Network TrainingComing soon
  11. #36Overfitting and Generalization: When Machine Learning MemorizesComing soon
  12. #37Building a Neural Language Model: Embeddings + MLPComing soon

Part 5Attention

  1. #38What Is Attention? How LLMs Read ContextComing soon
  2. #39Query, Key, Value: The Three Roles in the Attention MechanismComing soon
  3. #40Attention by Hand: Scaled Dot-Product Attention Step by StepComing soon
  4. #41Attention Scaling: Why Divide by √d Before SoftmaxComing soon
  5. #42Causal Mask: How GPT Avoids Looking at Future TokensComing soon
  6. #43Self-Attention as Matrix Math: The Core of the TransformerComing soon
  7. #44Multi-Head Attention: Reading Context from Several ViewpointsComing soon
  8. #45What Do Attention Heads Look At? Visualizing LLM AttentionComing soon
  9. #46Positional Encoding: Transformer Positions with Sine and CosineComing soon
  10. #47RoPE (Rotary Position Embedding): How Modern LLMs Encode PositionComing soon
  11. #48Attention Complexity: Why Long Context Makes LLMs SlowComing soon
  12. #49Building an Attention-Only Language Model: A One-Layer TransformerComing soon

Part 6Building the Transformer

  1. #50Residual Connections: Why Deep Transformers Can Be TrainedComing soon
  2. #51Layer Normalization: Keeping the Transformer StableComing soon
  3. #52The Feed-Forward (MLP) Layer: The Other Half of a TransformerComing soon
  4. #53Building a Transformer Block: Attention + MLP + ResidualsComing soon
  5. #54Stacking Transformer Layers: What Depth Gives an LLMComing soon
  6. #55The Residual Stream: How Information Flows Through a TransformerComing soon
  7. #56Logits and the Unembedding Layer: From Vectors Back to TokensComing soon
  8. #57Counting LLM Parameters: Where Are GPT's Weights?Coming soon
  9. #58How GPT Works, End to End: From Input to Next-Token PredictionComing soon

Part 7Training

  1. #59LLM Training Data: Self-Supervised Next-Token PredictionComing soon
  2. #60Reading a Loss Curve: Diagnosing LLM TrainingComing soon
  3. #61Optimizers for LLM Training: From SGD to Adam and AdamWComing soon
  4. #62Learning Rate Schedules and Warmup in LLM TrainingComing soon
  5. #63Weight Initialization: Avoiding Vanishing and Exploding GradientsComing soon
  6. #64Dropout and Weight Decay: Regularizing Neural NetworksComing soon
  7. #65Train a Mini GPT in Your Browser: A Transformer from ScratchComing soon
  8. #66What Does an LLM Learn During Training? Stages of LearningComing soon
  9. #67Scaling Laws: Why Bigger LLMs Are SmarterComing soon
  10. #68Perplexity: Measuring Language Model QualityComing soon

Part 8Generation

  1. #69Greedy Decoding: The Simplest Way an LLM Generates TextComing soon
  2. #70What Is Temperature? Controlling LLM RandomnessComing soon
  3. #71Top-k and Top-p (Nucleus) Sampling in LLMsComing soon
  4. #72Repetition Penalty: Stopping an LLM from Repeating ItselfComing soon
  5. #73Beam Search: LLM Text Generation That Looks AheadComing soon
  6. #74KV Cache: How LLM Inference Gets FastComing soon
  7. #75EOS Token and Max Tokens: When Does an LLM Stop?Coming soon
  8. #76Structured Output and JSON Mode: Constraining an LLM with a GrammarComing soon

Part 9From Predictor to Assistant

  1. #77What Is Pre-training? How an LLM Foundation Model Is MadeComing soon
  2. #78What Is Fine-Tuning? Teaching an LLM Something NewComing soon
  3. #79Instruction Tuning: How an LLM Becomes a Chat AssistantComing soon
  4. #80RLHF: Reinforcement Learning from Human Feedback and LLM AlignmentComing soon
  5. #81What Is LoRA? Fine-Tuning an LLM with Few ParametersComing soon
  6. #82System Prompts and In-Context Learning in LLMsComing soon
  7. #83Few-Shot Prompting: Why Prompt Engineering WorksComing soon
  8. #84Chain of Thought and Reasoning Models: LLMs That Think FirstComing soon

Part 10Looking Inside

  1. #85Logit Lens: Reading an LLM's Middle Layers (Interpretability)Coming soon
  2. #86Can You Trust Attention Maps? LLM InterpretabilityComing soon
  3. #87Superposition and Features in LLMs: An Intro to Sparse AutoencodersComing soon
  4. #88LLM Circuits: Tracing Computation with Mechanistic InterpretabilityComing soon
  5. #89Why Do LLMs Hallucinate? An Explanation from the MechanismComing soon
  6. #90Memorization vs Generalization in LLMs: GrokkingComing soon

Part 11Efficient and Practical

  1. #91What Is Quantization? Making LLMs Smaller and FasterComing soon
  2. #92Knowledge Distillation: How Small Language Models (SLMs) Are MadeComing soon
  3. #93Running a Local LLM on CPU: How llama.cpp and GGUF WorkComing soon
  4. #94How RAG Works: Retrieval-Augmented Generation for LLMsComing soon
  5. #95AI Agents and Tool Use: How LLM Function Calling WorksComing soon
  6. #96Multimodal LLMs: Turning Images and Audio into TokensComing soon
  7. #97Long Context: How LLMs Handle Very Long InputsComing soon

Part 12Wrap-Up

  1. #98LLM Evaluation and Benchmarks: How to Measure and Read ThemComing soon
  2. #99LLM Limitations and AI Safety: What the Mechanism Tells UsComing soon
  3. #100Build GPT from Scratch, the Finale: Every Step of an LLM by HandComing soon