policy-gradient/GRPO-Zero

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1897 stars · 99 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A minimal from-scratch Python implementation of DeepSeek's GRPO (Group Relative Policy Optimization) algorithm for reinforcement learning training of large language models, depending only on PyTorch and tokenizers. It includes DAPO improvements like token-level policy gradient loss and KL divergence removal, and runs on a single 24-48GB GPU.

Use cases

• train an LLM with GRPO reinforcement learning from scratch
• run RLHF-style policy gradient training on a single consumer GPU
• learn how GRPO works by reading a minimal implementation
• fine-tune Qwen2.5 models on reasoning tasks with reinforcement learning
• train LLMs without transformers or vLLM dependencies
• experiment with DAPO improvements to GRPO

When to choose

• you want a minimal, dependency-light GRPO implementation that fits in 24-48GB VRAM
• you want to understand or modify the GRPO algorithm internals
• you're training small models on verifiable-reward tasks like CountDown

When to avoid

• you need a production-scale RLHF pipeline for large models
• you want built-in support for many model architectures or distributed training
• you need a turnkey fine-tuning framework with extensive configs

Facets

library · maturity active

llm-training reinforcement-learning machine-learning large-language-models reinforcement-learning deep-learning machine-learning python grpo rlhf policy-gradient deepseek-r1 pytorch from-scratch low-vram dapo gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="policy-gradient/GRPO-Zero")