# raiyanyahya/how-to-train-your-gpt

Build a modern LLM from scratch. Every line commented. Explained like we are five.

Repository: https://github.com/raiyanyahya/how-to-train-your-gpt
Canonical: https://ross.abutalabs.com/products/how-to-train-your-gpt
Language: Jupyter Notebook
License: MIT
License Family: permissive
Topics: attention-mechanism, deep-learning, educational, from-scratch, gpt, language-model, llama, llm, machine-learning, natural-language-processing, python, pytorch, tokenisation, transformers, tutorial
Last push: 2026-07-15T17:23:52+00:00

## Health v2 (maintenance only)
Score: 55/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 92, release rhythm 35, longevity 8
- inputs: {"age_days": 122, "days_push": 49, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases, young
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 3152, forks 384 (observed 2026-08-28T04:07:46.137827+00:00)

## What it is
A 12-chapter educational repository that teaches how to build, train, and run a modern LLaMA-style GPT language model from scratch in Python/PyTorch, with every line of code commented. It includes 28 standalone topic explainers covering attention, RoPE, RMSNorm, SwiGLU, KV cache, and more.

## Use cases
- learn how transformers and attention work from scratch
- build a GPT-style language model in PyTorch
- understand LLaMA architecture internals like RoPE and KV cache
- study a fully commented LLM training loop
- teach myself deep learning without prior ML experience
- run a from-scratch language model training in Colab

## When to choose
- you want to understand LLM internals line by line rather than call APIs
- you have basic Python skills but no ML background
- you prefer analogies and step-by-step explanations over dense papers
- you want runnable notebooks for hands-on learning

## When to avoid
- you need a production-ready LLM framework or inference server
- you want to fine-tune existing models efficiently at scale
- you need battle-tested, optimized training code for real workloads

## Facets
- artifact type: learning-resource
- maturity: active
- function: llm-training, machine-learning, deep-learning, nlp, transformers
- domain: large-language-models, deep-learning, machine-learning, tutorials, education
- platform: python, cross-platform
- tags: pytorch, gpt, llama, from-scratch, tutorial, jupyter-notebook, attention-mechanism, tokenization, educational, natural-language-processing, gpu

## Member repositories
- raiyanyahya/how-to-train-your-gpt (main) score 55

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:07:46.137827+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T07:25:41.537124+00:00, confidence not recorded.
  - readme: https://github.com/raiyanyahya/how-to-train-your-gpt (fetched 2026-08-28T04:07:46.137827+00:00, sha 8d53d722bbc7)
- Data as of 2026-08-30T08:39:29.467469+00:00.