# karpathy/nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.

Repository: https://github.com/karpathy/nanoGPT
Canonical: https://ross.abutalabs.com/products/nanogpt
Language: Python
License: MIT
License Family: permissive
Last push: 2025-11-12T19:52:34+00:00

## Health v2 (maintenance only)
Score: 54/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 51, release rhythm 35, longevity 96
- inputs: {"age_days": 1345, "days_push": 294, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 62511, forks 10764 (observed 2026-08-28T04:12:19.120359+00:00)

## What it is
A minimal, readable repository for training and finetuning medium-sized GPT models, containing a ~300-line training loop and a ~300-line GPT model definition in plain PyTorch. It can reproduce GPT-2 (124M) on OpenWebText and load OpenAI's pretrained GPT-2 checkpoints, though it is now deprecated in favor of nanochat.

## Use cases
- train a small GPT model from scratch on my own text data
- learn how GPT training works by reading simple code
- finetune a pretrained GPT-2 checkpoint on a custom dataset
- reproduce GPT-2 124M training on OpenWebText
- train a character-level language model on Shakespeare
- understand transformer model implementation in PyTorch

## When to choose
- you want a minimal, hackable codebase for training or finetuning GPT-style models
- you are learning how LLM training pipelines work end to end
- you need a readable reference implementation of a GPT model in PyTorch

## When to avoid
- you need a maintained, production-grade training framework
- you want the latest features and improvements - use nanochat instead
- you need multi-node distributed training at scale or modern LLM techniques

## Facets
- artifact type: learning-resource
- maturity: abandoned
- function: llm-training, deep-learning, machine-learning
- domain: large-language-models, deep-learning, machine-learning, education
- platform: python
- tags: gpt, transformer, pytorch, training-from-scratch, finetuning, educational, minimal-implementation, deprecated, gpu, linux, macos

## Member repositories
- karpathy/nanoGPT (main) score 54

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:12:19.120359+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T16:17:27.888756+00:00, confidence not recorded.
  - readme: https://github.com/karpathy/nanoGPT (fetched 2026-08-28T04:12:19.120359+00:00, sha ddf78a0746d5)
- Data as of 2026-08-30T08:39:29.467469+00:00.