huggingface/alignment-handbook

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

5671 stars · 490 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

A collection of robust training recipes and scripts from Hugging Face for aligning large language models with human and AI preferences, covering SFT, DPO, ORPO, and RLHF pipelines. It includes reproducible recipes behind models like Zephyr and SmolLM, along with associated datasets and evaluation guidance.

Use cases

• fine-tune an LLM with DPO
• run supervised fine-tuning on a chat dataset
• align a model with human preferences using RLHF
• reproduce the Zephyr training recipe
• train a small instruct model like SmolLM
• compare preference alignment methods like DPO vs KTO vs IPO

When to choose

• you want battle-tested, reproducible recipes for LLM post-training
• you're using the Hugging Face Transformers/TRL ecosystem
• you need proven recipes for SFT plus preference alignment (DPO, ORPO, RLHF)

When to avoid

• you only need inference or serving of LLMs
• you want a general-purpose training framework rather than opinionated recipes
• you're training non-language models

Facets

library · maturity active

llm-training machine-learning deep-learning large-language-models machine-learning deep-learning artificial-intelligence python rlhf dpo sft orpo fine-tuning transformers recipes zephyr gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="huggingface/alignment-handbook")