karpathy/nanochat

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

57548 stars · 8005 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

nanochat is a minimal, hackable full-stack LLM training harness that covers tokenization, pretraining, finetuning, evaluation, and inference on a single GPU node. It lets users train GPT-2-capability models for tens of dollars and chat with them via a CLI, with a single complexity dial controlling model scale.

Use cases

• train my own GPT-2 from scratch on a single GPU node
• learn how LLM pretraining and finetuning work end to end
• run a cheap ChatGPT-like model for under $100
• benchmark how fast I can train a GPT-2 quality model
• experiment with compute-optimal transformer hyperparameters
• fine-tune and evaluate a small language model
• chat with a model I trained myself over a CLI

When to choose

• you want a minimal, readable codebase covering the full LLM training pipeline
• you have access to a single 8xH100 GPU node and want to train small GPT models cheaply
• you are learning how tokenization, pretraining, SFT, evaluation, and inference fit together
• you want to compete on the GPT-2 speedrun leaderboard

When to avoid

• you need production-grade distributed training across many nodes
• you want to train frontier-scale models with advanced features like MoE or RLHF pipelines
• you need a polished inference server or managed API
• you have no GPU access, since the whole point is GPU training

Facets

framework · maturity active

llm-training llm-inference machine-learning deep-learning cli large-language-models deep-learning machine-learning artificial-intelligence developer-tools education python cli gpt-2 speedrun transformer pretraining finetuning single-node h100 scaling-laws chatgpt-clone educational tokenization gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="karpathy/nanochat")