# lsdefine/simple_GRPO

A very simple  GRPO implement for reproducing r1-like LLM thinking.

Repository: https://github.com/lsdefine/simple_GRPO
Canonical: https://ross.abutalabs.com/products/simple_grpo
Language: Python
License: Apache-2.0
License Family: permissive
Last push: 2025-11-21T01:03:17+00:00

## Health v2 (maintenance only)
Score: 44/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 53, release rhythm 35, longevity 40
- inputs: {"age_days": 570, "days_push": 286, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1702, forks 132 (observed 2026-08-28T04:05:24.401638+00:00)

## What it is
A minimal (~200 lines) Python implementation of GRPO reinforcement learning for training LLMs to develop r1-like reasoning, built on PyTorch and DeepSpeed. It decouples the reference model onto separate GPUs to save memory, enabling 7B model training on a single 80GB A800.

## Use cases
- train an LLM with GRPO reinforcement learning
- reproduce r1-style reasoning and the 'aha moment' in small models
- learn how GRPO works from a simple teaching codebase
- fine-tune Qwen models with RL on limited GPU memory
- experiment with KL penalties, regrouping, and multi-answer generation
- run reference model on a separate GPU or machine

## When to choose
- you want a small, hackable GRPO codebase to understand or modify
- you have limited GPU memory and need the reference model decoupled
- you want to quickly reproduce r1-like thinking on Qwen models
- you prefer minimal dependencies (torch + deepspeed, no ray)

## When to avoid
- you need a production-grade, feature-complete RLHF framework
- you want distributed multi-node training with ray-based orchestration
- you need built-in dataset handling and experiment tracking
- you have fewer than two GPUs

## Facets
- artifact type: library
- maturity: active
- function: llm-training, machine-learning, deep-learning
- domain: large-language-models, machine-learning, deep-learning
- platform: python
- tags: grpo, reinforcement-learning, rlhf, reasoning, deepspeed, minimal-implementation, r1-reproduction, gpu, linux

## Member repositories
- lsdefine/simple_GRPO (main) score 44

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:24.401638+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T03:37:40.647875+00:00, confidence not recorded.
  - readme: https://github.com/lsdefine/simple_GRPO (fetched 2026-08-28T04:05:24.401638+00:00, sha d04ed6399228)
- Data as of 2026-08-30T08:39:29.467469+00:00.