# datawhalechina/llms-from-scratch-cn

仅需Python基础，从0构建大语言模型；从0逐步构建GLM4\Llama3\RWKV6， 深入理解大模型原理

Repository: https://github.com/datawhalechina/llms-from-scratch-cn
Canonical: https://ross.abutalabs.com/products/llms-from-scratch-cn
Language: Jupyter Notebook
License: NOASSERTION
License Family: other
Topics: glm, llama, llm, rwkv, llms-from-scratch
Last push: 2026-03-26T04:32:32+00:00

## Health v2 (maintenance only)
Score: 59/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 74, release rhythm 35, longevity 65
- inputs: {"age_days": 917, "days_push": 160, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases, no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 4337, forks 595 (observed 2026-08-28T04:08:46.063649+00:00)

## What it is
A hands-on tutorial by Datawhale for building large language models from scratch in Python, covering tokenization, attention, pretraining, and fine-tuning. It includes step-by-step notebooks for implementing GLM4, Llama3, and RWKV6 architectures to deeply understand LLM internals.

## Use cases
- learn how large language models work by building one from scratch
- implement GPT-like LLM architecture in PyTorch
- understand attention mechanisms and transformer internals
- build GLM4, Llama3, or RWKV6 from zero
- study LLM pretraining and fine-tuning with runnable notebooks
- find a structured curriculum for learning LLM fundamentals

## When to choose
- you want an educational, code-first path to understanding LLM internals
- you have basic Python and PyTorch skills and want to implement models yourself
- you prefer step-by-step notebooks over reading papers alone
- you want to study specific architectures like GLM4, Llama3, or RWKV6

## When to avoid
- you need a production-ready LLM training framework
- you want to fine-tune or deploy existing large models rather than learn internals
- you need large-scale distributed training tooling
- you need a permissively licensed codebase for commercial reuse (license is non-standard)

## Facets
- artifact type: learning-resource
- maturity: active
- function: llm-training, machine-learning, deep-learning
- domain: large-language-models, deep-learning, tutorials, education
- platform: python
- tags: llms-from-scratch, glm4, llama3, rwkv6, jupyter-notebooks, chinese, datawhale, hands-on-tutorial

## Member repositories
- datawhalechina/llms-from-scratch-cn (main) score 59

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:08:46.063649+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T18:21:38.092687+00:00, confidence not recorded.
  - readme: https://github.com/datawhalechina/llms-from-scratch-cn (fetched 2026-08-28T04:08:46.063649+00:00, sha 688c205e0e45)
- Data as of 2026-08-30T08:39:29.467469+00:00.