datawhalechina/diy-llm resource

Health v2 · maintenance only

Flags: no_license

Adoption not part of the score

1278 stars · 131 forks observed · 2026-09-03

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A Chinese-language, code-driven course (adapted from Stanford CS336) for systematically building large language models from scratch. It covers tokenizers, Transformer architecture, MoE, CUDA/Triton GPU programming, distributed training, scaling laws, inference optimization, data engineering, and alignment (SFT/RLHF/GRPO) through six progressive hands-on assignments.

Use cases

• learn how large language models are built from scratch
• implement a BPE tokenizer and transformer in PyTorch
• learn CUDA and Triton GPU programming for LLMs
• understand distributed training and FSDP
• study scaling laws and compute-optimal training
• practice SFT and RLHF alignment with GRPO
• chinese-language alternative to stanford cs336

When to choose

• you want a code-driven, hands-on LLM full-stack curriculum in Chinese
• you already know Python, PyTorch, and deep learning basics and want to go deeper
• you want to cover everything from data engineering to GPU kernels and alignment in one course

When to avoid

• you need a production LLM training framework or library rather than a course
• you are a complete beginner without Python and deep learning fundamentals
• you need English-only materials

Facets

learning-resource · maturity active

llm-training llm-inference gpu-computing machine-learning deep-learning prompt-engineering large-language-models deep-learning machine-learning tutorials education gpu-computing python cross-platform course chinese-language cs336 cuda triton moe rlhf scaling-laws tokenizer distributed-training jupyter-notebook datawhale gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="datawhalechina/diy-llm")