lsdefine/simple_GRPO

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1702 stars · 132 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A minimal (~200 lines) Python implementation of GRPO reinforcement learning for training LLMs to develop r1-like reasoning, built on PyTorch and DeepSpeed. It decouples the reference model onto separate GPUs to save memory, enabling 7B model training on a single 80GB A800.

Use cases

• train an LLM with GRPO reinforcement learning
• reproduce r1-style reasoning and the 'aha moment' in small models
• learn how GRPO works from a simple teaching codebase
• fine-tune Qwen models with RL on limited GPU memory
• experiment with KL penalties, regrouping, and multi-answer generation
• run reference model on a separate GPU or machine

When to choose

• you want a small, hackable GRPO codebase to understand or modify
• you have limited GPU memory and need the reference model decoupled
• you want to quickly reproduce r1-like thinking on Qwen models
• you prefer minimal dependencies (torch + deepspeed, no ray)

When to avoid

• you need a production-grade, feature-complete RLHF framework
• you want distributed multi-node training with ray-based orchestration
• you need built-in dataset handling and experiment tracking
• you have fewer than two GPUs

Facets

library · maturity active

llm-training machine-learning deep-learning large-language-models machine-learning deep-learning python grpo reinforcement-learning rlhf reasoning deepspeed minimal-implementation r1-reproduction gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="lsdefine/simple_GRPO")