# datawhalechina/diy-llm

🎓 系统性大语言模型构建课程｜🛠️ 覆盖预训练数据工程、Tokenizer、Transformer、MoE、GPU 编程 (CUDA/Triton)、分布式训练、Scaling Laws、推理优化及对齐 (SFT/RLHF/GRPO)｜🚀 6 个渐进式作业 + 代码驱动，建立 LLM 全栈认知体系

Repository: https://github.com/datawhalechina/diy-llm
Canonical: https://ross.abutalabs.com/products/diy-llm
Homepage: https://datawhalechina.github.io/diy-llm/
Language: Jupyter Notebook
License Family: other
Topics: gpu-programming, llm, nlp, rl, sft, transformer
Last push: 2026-09-02T09:03:11+00:00

## Health v2 (maintenance only)
Score: 79/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 100, release rhythm 86, longevity 20
- inputs: {"age_days": 282, "days_push": 0, "days_rel": 16, "gap_med": 68, "n_releases_24m": 2}
- flags: no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1278, forks 131 (observed 2026-09-03T02:15:11.175270+00:00)

## What it is
A Chinese-language, code-driven course (adapted from Stanford CS336) for systematically building large language models from scratch. It covers tokenizers, Transformer architecture, MoE, CUDA/Triton GPU programming, distributed training, scaling laws, inference optimization, data engineering, and alignment (SFT/RLHF/GRPO) through six progressive hands-on assignments.

## Use cases
- learn how large language models are built from scratch
- implement a BPE tokenizer and transformer in PyTorch
- learn CUDA and Triton GPU programming for LLMs
- understand distributed training and FSDP
- study scaling laws and compute-optimal training
- practice SFT and RLHF alignment with GRPO
- chinese-language alternative to stanford cs336

## When to choose
- you want a code-driven, hands-on LLM full-stack curriculum in Chinese
- you already know Python, PyTorch, and deep learning basics and want to go deeper
- you want to cover everything from data engineering to GPU kernels and alignment in one course

## When to avoid
- you need a production LLM training framework or library rather than a course
- you are a complete beginner without Python and deep learning fundamentals
- you need English-only materials

## Facets
- artifact type: learning-resource
- maturity: active
- function: llm-training, llm-inference, gpu-computing, machine-learning, deep-learning, prompt-engineering
- domain: large-language-models, deep-learning, machine-learning, tutorials, education, gpu-computing
- platform: python, cross-platform
- tags: course, chinese-language, cs336, cuda, triton, moe, rlhf, scaling-laws, tokenizer, distributed-training, jupyter-notebook, datawhale, gpu

## Member repositories
- datawhalechina/diy-llm (main) score 79

## Provenance
- Observed fields: from GitHub, fetched 2026-09-03T02:15:11.175270+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T05:07:51.862951+00:00, confidence not recorded.
  - readme: https://github.com/datawhalechina/diy-llm (fetched 2026-09-03T02:15:11.175270+00:00, sha 1213b67ad584)
  - homepage: https://datawhalechina.github.io/diy-llm/ (fetched 2026-08-29T12:19:00.989102+00:00, sha dc7b04b11a57)
- Data as of 2026-08-30T08:39:29.467469+00:00.