karpathy/build-nanogpt resource

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

5461 stars · 886 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

A step-by-step video lecture and companion code repository by Andrej Karpathy that builds a GPT-2 (124M) reproduction from an empty file, with clean git commits documenting each stage. It is an educational resource for learning how transformer language models are implemented and trained, not a production training framework.

Use cases

• learn how GPT-2 works by building it from scratch
• follow a guided video lecture on training a small language model
• walk through git commits to see a transformer implemented incrementally
• reproduce GPT-2 124M training on a cloud GPU for about $10
• study PyTorch code for transformer training
• prepare for understanding LLM finetuning and SFT

When to choose

• you want to deeply understand how GPT-style language models are built and trained
• you prefer learning through annotated, incremental code with video explanations
• you want a minimal, readable reference implementation of GPT-2 training

When to avoid

• you need a production-ready LLM training framework with distributed scaling and finetuning
• you want a chat model - this covers pretraining only, not instruction tuning
• you need maintained software with a license or active feature development

Facets

learning-resource · maturity stable

llm-training deep-learning machine-learning large-language-models deep-learning tutorials machine-learning python cross-platform educational gpt-2 from-scratch video-lecture nanogpt transformers gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="karpathy/build-nanogpt")