angelos-p/llm-from-scratch resource

Health v2 · maintenance only

Flags: no_releases young no_license

Adoption not part of the score

3380 stars · 367 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

A hands-on workshop repository where learners write every component of a GPT training pipeline (tokenizer, transformer, training loop, generation) from scratch in PyTorch. It targets a ~10M parameter model that trains on a laptop in under an hour, inspired by Karpathy's nanoGPT.

Use cases

• learn how transformers and GPT work by building one from scratch
• train a small language model on my laptop
• understand tokenization, attention, and training loops hands-on
• teach an LLM workshop session
• generate Shakespeare-like text with a model I wrote myself
• get started with LLMs without prior ML experience

When to choose

• you want to deeply understand GPT internals by writing every piece yourself
• you have no GPU cluster and want a model that trains on a laptop in under an hour
• you're a beginner with Python skills but no ML background
• you want a single-session guided workshop format

When to avoid

• you need to train production-scale or GPT-2-sized models
• you want a ready-made library or framework rather than guided exercises
• you need fine-tuning, RLHF, or instruction-tuning pipelines
• you want prebuilt model weights or inference tooling

Facets

learning-resource · maturity active

llm-training machine-learning deep-learning large-language-models deep-learning machine-learning education tutorials python windows cross-platform gpt transformer workshop nanogpt pytorch hands-on from-scratch education macos linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="angelos-p/llm-from-scratch")