datawhalechina/llms-from-scratch-cn resource

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

4337 stars · 595 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

A hands-on tutorial by Datawhale for building large language models from scratch in Python, covering tokenization, attention, pretraining, and fine-tuning. It includes step-by-step notebooks for implementing GLM4, Llama3, and RWKV6 architectures to deeply understand LLM internals.

Use cases

• learn how large language models work by building one from scratch
• implement GPT-like LLM architecture in PyTorch
• understand attention mechanisms and transformer internals
• build GLM4, Llama3, or RWKV6 from zero
• study LLM pretraining and fine-tuning with runnable notebooks
• find a structured curriculum for learning LLM fundamentals

When to choose

• you want an educational, code-first path to understanding LLM internals
• you have basic Python and PyTorch skills and want to implement models yourself
• you prefer step-by-step notebooks over reading papers alone
• you want to study specific architectures like GLM4, Llama3, or RWKV6

When to avoid

• you need a production-ready LLM training framework
• you want to fine-tune or deploy existing large models rather than learn internals
• you need large-scale distributed training tooling
• you need a permissively licensed codebase for commercial reuse (license is non-standard)

Facets

learning-resource · maturity active

llm-training machine-learning deep-learning large-language-models deep-learning tutorials education python llms-from-scratch glm4 llama3 rwkv6 jupyter-notebooks chinese datawhale hands-on-tutorial

Member repositories

For agents

markdown · JSON · MCP: product_card(name="datawhalechina/llms-from-scratch-cn")