Jiayi-Pan/TinyZero

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

13223 stars · 1577 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

TinyZero is a minimal reproduction of DeepSeek R1-Zero, showing that a 3B base language model can develop self-verification and search abilities through reinforcement learning on countdown and multiplication tasks. It is built on top of the veRL library and is now deprecated in favor of veRL itself.

Use cases

• reproduce deepseek r1-zero training on a small budget
• train a small language model with reinforcement learning to learn reasoning
• run RL experiments on countdown and multiplication tasks
• observe emergent self-verification in a 3B base model
• learn how to use veRL for LLM RL training
• experiment with RL post-training of Qwen models

When to choose

• you want a cheap, minimal reference implementation of R1-Zero-style RL training
• you are studying how reasoning abilities emerge via RL in small base models
• you want a starting point built on veRL for reasoning RL experiments

When to avoid

• you need an actively maintained RL training framework - use veRL directly instead
• you want production LLM training infrastructure
• you cannot access GPUs - training requires multi-GPU setups for 3B+ models

Facets

library · maturity abandoned

llm-training reinforcement-learning machine-learning large-language-models deep-learning reinforcement-learning artificial-intelligence python deepseek-r1-zero verl reasoning rl-training research-reproduction countdown-task vllm gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="Jiayi-Pan/TinyZero")