CarperAI/trlx

Health v2 · maintenance only

Adoption not part of the score

4755 stars · 486 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

trlX is a distributed training framework for fine-tuning large language models with reinforcement learning from human feedback (RLHF), supporting PPO and ILQL algorithms. It provides Accelerate-backed trainers for models up to 20B parameters and NVIDIA NeMo-backed trainers for larger models.

Use cases

• fine-tune an LLM with RLHF using a reward function
• train a 20B parameter language model with PPO
• apply ILQL to a reward-labeled dataset
• distributed RL fine-tuning of GPT-NeoX or Flan-T5
• align a language model with human preferences

When to choose

• you need distributed RLHF training for large causal or T5-based language models
• you want PPO or ILQL implementations on top of Hugging Face models
• you need to scale fine-tuning beyond 20B parameters with NeMo

When to avoid

• you only need supervised fine-tuning without reinforcement learning
• you want actively maintained tooling - the project is in maintenance mode and TRL is the more active successor
• you work outside the PyTorch/Hugging Face ecosystem

Facets

library · maturity maintenance

llm-training machine-learning reinforcement-learning large-language-models reinforcement-learning machine-learning deep-learning python rlhf ppo ilql distributed-training pytorch huggingface nemo fine-tuning gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="CarperAI/trlx")