qibin0506/Cortex resource

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

2689 stars · 209 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Cortex is an open-source educational project that walks through building a large language model from scratch, covering the full training lifecycle: pretraining, midtraining, SFT, DPO, and PPO-based RLHF. It features a lightweight 0.1B-parameter MoE model, thinking-mode control via /think and /no think tags, and has been validated on domestic Chinese MLU370 chips.

Use cases

• learn how to pretrain an LLM from scratch
• understand the full RLHF pipeline with DPO and PPO
• train a small MoE model on limited hardware
• study instruction tuning and preference alignment code
• run a tiny chat model locally with a web UI
• adapt LLM training to domestic accelerators like MLU370

When to choose

• you want a complete, decoupled, end-to-end LLM training codebase to learn from
• you need an extremely lightweight MoE reference implementation
• you want runnable examples of pretrain, SFT, DPO, and PPO stages
• you are exploring LLM-as-Judge reward signals for PPO

When to avoid

• you need a production-grade LLM or state-of-the-art model quality
• you want a turnkey fine-tuning framework rather than educational code
• you require large-scale multi-node training support out of the box
• you need broad model architecture compatibility

Facets

learning-resource · maturity active

llm-training machine-learning deep-learning rag chatbot large-language-models deep-learning machine-learning education tutorials python llm-from-scratch pretraining sft dpo ppo rlhf moe deepspeed chinese modelscope linux gpu docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="qibin0506/Cortex")