alibaba/ROLL

Health v2 · maintenance only

Adoption not part of the score

3374 stars · 309 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

ROLL is an open-source reinforcement learning library from Alibaba for training large language models at scale, supporting algorithms like PPO and GRPO. It uses a Ray-based distributed architecture integrating Megatron-Core, SGLang, and vLLM for efficient training and inference across large GPU clusters.

Use cases

• train an LLM with RLHF for human preference alignment
• run RLVR training to improve model reasoning
• fine-tune an LLM with PPO or GRPO on a GPU cluster
• train multi-turn agentic RL policies
• scale reinforcement learning post-training to thousands of GPUs
• on-policy distillation of large language models

When to choose

• you need large-scale RL post-training for LLMs with flexible multi-node GPU allocation
• you want built-in support for PPO, GRPO, multi-task RL, and agentic interaction scenarios
• you want to combine Megatron-Core training with vLLM/SGLang inference engines

When to avoid

• you only need simple supervised fine-tuning without reinforcement learning
• you lack access to substantial GPU resources
• you need a lightweight single-GPU RLHF tool with minimal setup

Facets

library · maturity active

llm-training reinforcement-learning llm-inference agent-framework gpu-computing large-language-models reinforcement-learning machine-learning gpu-computing python cloud rlhf rlvr ppo grpo ray vllm sglang megatron-core distributed-training agentic-rl ai-agents gpu linux docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="alibaba/ROLL")