ChenmienTan/RL2

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1316 stars · 134 forks observed · 2026-10-10

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

RL2 (Ray Less Reinforcement Learning) is a concise Python library for post-training large language models with reinforcement learning, SFT, RM, and DPO. It offers a clear, low-abstraction implementation while scaling via FSDP and Megatron 3D/5D parallelism and multi-turn rollout with SGLang.

Use cases

• train an LLM with reinforcement learning post-training
• run SFT fine-tuning on custom datasets
• train reward models or run DPO alignment
• quickly prototype a new RL algorithm for LLMs
• scale RL training across multiple GPUs with parallelism
• run multi-turn agentic rollouts with SGLang

When to choose

• you want a readable, low-abstraction RLHF/RL training codebase to learn from or modify
• you need quick experiments on RL post-training algorithms
• you need scalable training with FSDP or Megatron parallelism
• you want agentic environment rollouts via GEM and SGLang

When to avoid

• you need a fully turnkey enterprise training platform with extensive abstractions
• you require LoRA/PEFT support, which is not yet implemented
• your workflow depends on Ray-based orchestration

Facets

library · maturity active

llm-training machine-learning gpu-computing benchmarking large-language-models machine-learning deep-learning reinforcement-learning python reinforcement-learning post-training sft dpo rlhf fsdp megatron sglang distributed-training agentic-environments gpu docker linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="ChenmienTan/RL2")