OpenRLHF/OpenRLHF

Health v2 · maintenance only

Adoption not part of the score

9956 stars · 1005 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

OpenRLHF is a high-performance, production-ready open-source RLHF framework built on Ray + vLLM + DeepSpeed for scalable reinforcement learning from human feedback. It supports state-of-the-art RL algorithms (PPO, GRPO, REINFORCE++, RLOO), SFT, reward modeling, DPO, and vision-language model training through a unified agent-based pipeline.

Use cases

• train llm with rlhf ppo
• run grpo reinforcement learning fine-tuning
• fine-tune a 70b model with reinforcement learning from human feedback
• train vision-language model with rl
• sft and reward model training pipeline
• async rl training with vllm rollout
• multi-turn agent reinforcement learning training

When to choose

• you need scalable RLHF/RL post-training for large language models on multi-GPU clusters
• you want to switch between PPO, GRPO, REINFORCE++, and RLOO with a single flag
• you need vLLM-accelerated generation and DeepSpeed ZeRO-3 training from HuggingFace checkpoints
• you want to train VLMs or multi-turn agents end-to-end with RL

When to avoid

• you only need simple supervised fine-tuning without RL
• you lack multi-GPU infrastructure or distributed training experience
• you need a lightweight single-GPU hobbyist trainer
• you prefer a non-Python training stack

Facets

framework · maturity active

llm-training reinforcement-learning machine-learning gpu-computing cli reinforcement-learning large-language-models machine-learning gpu-computing python rlhf ppo grpo vllm ray deepspeed dpo vlm-training distributed-training agent-rl linux gpu docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="OpenRLHF/OpenRLHF")