verl-project/verl

Health v2 · maintenance only

Adoption not part of the score

23145 stars · 4448 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

verl (Volcano Engine Reinforcement Learning) is a flexible, production-ready RL post-training library for large language models, open-sourced from ByteDance Seed's HybridFlow paper. It implements a hybrid-controller programming model that lets users express RLHF dataflows like PPO and GRPO in a few lines while integrating with FSDP, Megatron-LM, vLLM, and SGLang for state-of-the-art training and rollout throughput.

Use cases

• train LLMs with RLHF using PPO or GRPO
• run RL post-training on my own GPUs with vLLM rollouts
• implement a custom RL algorithm for language model fine-tuning
• scale RLHF training across a multi-GPU cluster
• fine-tune a HuggingFace model with reinforcement learning
• train a draft model for speculative decoding
• run RL post-training on AMD ROCm GPUs

When to choose

• you need production-grade RLHF/RL post-training for LLMs with high throughput
• you want flexible algorithm experimentation with PPO, GRPO, or custom RL dataflows
• you need to combine existing training (FSDP, Megatron-LM) and inference (vLLM, SGLang) engines efficiently
• you need flexible GPU placement across different cluster sizes or heterogeneous hardware

When to avoid

• you only need supervised fine-tuning (SFT) without RL
• you want a simple one-command fine-tuning script rather than a configurable RL framework
• you are not working with large language models or GPU clusters
• you need a lightweight library without distributed training infrastructure

Facets

library · maturity active

llm-training reinforcement-learning machine-learning gpu-computing large-language-models reinforcement-learning machine-learning gpu-computing python rlhf post-training ppo grpo vllm sglang fsdp megatron-lm bytedance hybridflow gpu linux docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="verl-project/verl")