huggingface/trl

Health v2 · maintenance only

Adoption not part of the score

19155 stars · 2933 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

TRL is a Hugging Face library for post-training transformer language models with techniques like Supervised Fine-Tuning (SFT), GRPO, DPO, KTO, reward modeling, and knowledge distillation. It builds on the Transformers ecosystem and scales from single GPUs to multi-node clusters via Accelerate, DeepSpeed, PEFT/LoRA, and vLLM.

Use cases

• fine-tune an LLM with SFT on my own dataset
• train a language model with RLHF or DPO preference optimization
• run GRPO reinforcement learning to improve model reasoning
• post-train a foundation model with LoRA on a single GPU
• distill a teacher LLM into a smaller student model
• train a reward model for preference tuning
• scale LLM fine-tuning across multiple GPUs with DeepSpeed

When to choose

• You want to fine-tune or post-train transformer LLMs within the Hugging Face Transformers ecosystem
• You need modern alignment methods like SFT, DPO, GRPO, or KTO with a consistent trainer API
• You want parameter-efficient training (LoRA/QLoRA) or multi-GPU/multi-node scaling out of the box

When to avoid

• You are training non-transformer models or non-language modalities
• You need a framework-agnostic trainer outside PyTorch/Hugging Face
• You only need inference or serving of models rather than training

Facets

library · maturity active

llm-training machine-learning deep-learning cli large-language-models machine-learning reinforcement-learning artificial-intelligence python windows rlhf sft dpo grpo post-training huggingface transformers fine-tuning peft vllm gpu linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="huggingface/trl")