karpathy/llm.c

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

30879 stars · 3744 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

llm.c is a pure C/CUDA implementation for training large language models like GPT-2 and GPT-3, with no dependency on PyTorch or Python. It includes a compact ~1,000-line CPU reference implementation and a faster-than-PyTorch CUDA training path.

Use cases

• train a GPT-2 model from scratch in pure C
• learn how LLM training works at the CUDA level
• pretrain GPT-2/GPT-3 scale models without PyTorch
• study a minimal readable LLM training implementation
• benchmark custom CUDA training against PyTorch
• run LLM pretraining on a single GPU with fp32

When to choose

• you want to understand or modify every detail of LLM training in C/CUDA
• you need a lightweight training stack without PyTorch's overhead
• you're reproducing GPT-2/GPT-3 pretraining on NVIDIA GPUs
• you're learning CUDA through a real, high-performance codebase

When to avoid

• you need a production training framework with broad model support
• you want fine-tuning, RLHF, or inference features out of the box
• you don't have NVIDIA GPUs and need multi-backend support
• you prefer Python ecosystem tooling and integrations

Facets

library · maturity active

llm-training deep-learning machine-learning gpu-computing large-language-models deep-learning machine-learning gpu-computing cpp c cuda gpt-2 gpt-3 pretraining educational no-pytorch karpathy linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="karpathy/llm.c")