karpathy/nanoGPT resource

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

62511 stars · 10764 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

A minimal, readable repository for training and finetuning medium-sized GPT models, containing a ~300-line training loop and a ~300-line GPT model definition in plain PyTorch. It can reproduce GPT-2 (124M) on OpenWebText and load OpenAI's pretrained GPT-2 checkpoints, though it is now deprecated in favor of nanochat.

Use cases

• train a small GPT model from scratch on my own text data
• learn how GPT training works by reading simple code
• finetune a pretrained GPT-2 checkpoint on a custom dataset
• reproduce GPT-2 124M training on OpenWebText
• train a character-level language model on Shakespeare
• understand transformer model implementation in PyTorch

When to choose

• you want a minimal, hackable codebase for training or finetuning GPT-style models
• you are learning how LLM training pipelines work end to end
• you need a readable reference implementation of a GPT model in PyTorch

When to avoid

• you need a maintained, production-grade training framework
• you want the latest features and improvements - use nanochat instead
• you need multi-node distributed training at scale or modern LLM techniques

Facets

learning-resource · maturity abandoned

llm-training deep-learning machine-learning large-language-models deep-learning machine-learning education python gpt transformer pytorch training-from-scratch finetuning educational minimal-implementation deprecated gpu linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="karpathy/nanoGPT")