# huggingface/trl

Train transformer language models with reinforcement learning.

Repository: https://github.com/huggingface/trl
Canonical: https://ross.abutalabs.com/products/trl
Homepage: http://hf.co/docs/trl
Language: Python
License: Apache-2.0
License Family: permissive
Last push: 2026-08-26T20:03:22+00:00

## Health v2 (maintenance only)
Score: 99/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 99, longevity 100
- inputs: {"age_days": 2350, "days_push": 7, "days_rel": 7, "gap_med": 11.0, "n_releases_24m": 57}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 19155, forks 2933 (observed 2026-08-28T04:11:27.461331+00:00)

## What it is
TRL is a Hugging Face library for post-training transformer language models with techniques like Supervised Fine-Tuning (SFT), GRPO, DPO, KTO, reward modeling, and knowledge distillation. It builds on the Transformers ecosystem and scales from single GPUs to multi-node clusters via Accelerate, DeepSpeed, PEFT/LoRA, and vLLM.

## Use cases
- fine-tune an LLM with SFT on my own dataset
- train a language model with RLHF or DPO preference optimization
- run GRPO reinforcement learning to improve model reasoning
- post-train a foundation model with LoRA on a single GPU
- distill a teacher LLM into a smaller student model
- train a reward model for preference tuning
- scale LLM fine-tuning across multiple GPUs with DeepSpeed

## When to choose
- You want to fine-tune or post-train transformer LLMs within the Hugging Face Transformers ecosystem
- You need modern alignment methods like SFT, DPO, GRPO, or KTO with a consistent trainer API
- You want parameter-efficient training (LoRA/QLoRA) or multi-GPU/multi-node scaling out of the box

## When to avoid
- You are training non-transformer models or non-language modalities
- You need a framework-agnostic trainer outside PyTorch/Hugging Face
- You only need inference or serving of models rather than training

## Facets
- artifact type: library
- maturity: active
- function: llm-training, machine-learning, deep-learning, cli
- domain: large-language-models, machine-learning, reinforcement-learning, artificial-intelligence
- platform: python, windows
- tags: rlhf, sft, dpo, grpo, post-training, huggingface, transformers, fine-tuning, peft, vllm, gpu, linux, macos

## Member repositories
- huggingface/trl (main) score 99

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:11:27.461331+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T17:00:28.482975+00:00, confidence not recorded.
  - readme: https://github.com/huggingface/trl (fetched 2026-08-28T04:11:27.461331+00:00, sha 4807cbb081fa)
  - homepage: http://hf.co/docs/trl (fetched 2026-08-29T07:58:51.682587+00:00, sha bde8532f3e6d)
  - site_page: https://huggingface.co/docs (fetched 2026-08-29T07:58:51.685412+00:00, sha bdec26667b98)
  - site_page: https://huggingface.co/docs/trl/installation (fetched 2026-08-29T07:58:51.689572+00:00, sha 6ab19072d32e)
  - site_page: https://huggingface.co/docs/trl/quickstart (fetched 2026-08-29T07:58:51.691383+00:00, sha 01a41386510e)
  - site_page: https://huggingface.co/docs/trl/usage_stats (fetched 2026-08-29T07:58:51.693283+00:00, sha 539ae35bc43f)
  - site_page: https://huggingface.co/docs/trl/chat_templates (fetched 2026-08-29T07:58:51.695053+00:00, sha 97c6c183ad73)
  - site_page: https://huggingface.co/docs/trl/dataset_formats (fetched 2026-08-29T07:58:51.697176+00:00, sha b9155d8286b5)
  - site_page: https://huggingface.co/docs/trl/paper_index (fetched 2026-08-29T07:58:51.699815+00:00, sha d84fa056581e)
  - registry_pypi: https://pypi.org/pypi/trl/json (fetched 2026-08-29T07:58:51.703299+00:00, sha 6e063225592c)
- Data as of 2026-08-30T08:39:29.467469+00:00.