# srush/LLM-Training-Puzzles

What would you do with 1000 H100s...

Repository: https://github.com/srush/LLM-Training-Puzzles
Canonical: https://ross.abutalabs.com/products/llm-training-puzzles
Language: Jupyter Notebook
License: MIT
License Family: permissive
Topics: llm, puzzles
Last push: 2024-01-10T17:11:43+00:00

## Health v2 (maintenance only)
Score: 29/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 0, release rhythm 35, longevity 83
- inputs: {"age_days": 1165, "days_push": 966, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1190, forks 72 (observed 2026-08-28T04:03:55.983773+00:00)

## What it is
A collection of 8 challenging Jupyter notebook puzzles about training large language models on many GPUs, teaching memory efficiency and compute pipelining primitives. It is part of Sasha Rush's series of educational puzzle repositories and is designed to run in Google Colab.

## Use cases
- learn how to train LLMs across thousands of GPUs
- practice distributed training primitives hands-on
- understand memory efficiency in multi-GPU training
- learn compute pipelining for neural network training
- teach a course on large-scale model training
- self-study exercises on multi-GPU deep learning

## When to choose
- you want interactive, hands-on exercises for distributed LLM training
- you are teaching or self-studying large-scale GPU training concepts
- you enjoyed the other srush puzzle series (gpu-puzzles, tensor-puzzles, etc.)

## When to avoid
- you need production distributed training code or a framework
- you want a turnkey solution for training LLMs
- you are a beginner without Python and deep learning basics

## Facets
- artifact type: learning-resource
- maturity: stable
- function: llm-training, gpu-computing, developer-tools
- domain: large-language-models, deep-learning, tutorials, education
- platform: python, cross-platform
- tags: puzzles, distributed-training, notebook, colab, hands-on-learning, gpu-distributed, gpu

## Member repositories
- srush/LLM-Training-Puzzles (main) score 29

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:03:55.983773+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T06:23:06.062870+00:00, confidence not recorded.
  - readme: https://github.com/srush/LLM-Training-Puzzles (fetched 2026-08-28T04:03:55.983773+00:00, sha 1edad80d3735)
- Data as of 2026-08-30T08:39:29.467469+00:00.