srush/LLM-Training-Puzzles resource

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1190 stars · 72 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A collection of 8 challenging Jupyter notebook puzzles about training large language models on many GPUs, teaching memory efficiency and compute pipelining primitives. It is part of Sasha Rush's series of educational puzzle repositories and is designed to run in Google Colab.

Use cases

• learn how to train LLMs across thousands of GPUs
• practice distributed training primitives hands-on
• understand memory efficiency in multi-GPU training
• learn compute pipelining for neural network training
• teach a course on large-scale model training
• self-study exercises on multi-GPU deep learning

When to choose

• you want interactive, hands-on exercises for distributed LLM training
• you are teaching or self-studying large-scale GPU training concepts
• you enjoyed the other srush puzzle series (gpu-puzzles, tensor-puzzles, etc.)

When to avoid

• you need production distributed training code or a framework
• you want a turnkey solution for training LLMs
• you are a beginner without Python and deep learning basics

Facets

learning-resource · maturity stable

llm-training gpu-computing developer-tools large-language-models deep-learning tutorials education python cross-platform puzzles distributed-training notebook colab hands-on-learning gpu-distributed gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="srush/LLM-Training-Puzzles")