lucidrains/self-rewarding-lm-pytorch

Health v2 · maintenance only

Adoption not part of the score

1411 stars · 70 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A PyTorch library implementing the Self-Rewarding Language Model training framework from MetaAI, along with the SPIN training method. It provides trainers that combine supervised fine-tuning, self-rewarding, and direct preference optimization (DPO) for language models.

Use cases

• train a self-rewarding language model in pytorch
• implement DPO fine-tuning for an LLM
• run SPIN training on a transformer
• fine-tune a language model with self-generated rewards
• reproduce the MetaAI self-rewarding LM paper
• build a preference optimization training pipeline

When to choose

• you want a lightweight, hackable PyTorch implementation of self-rewarding LM or SPIN training
• you are doing research on preference optimization methods like DPO and SPIN
• you already have a transformer model and want to add self-rewarding fine-tuning stages

When to avoid

• you need a production-grade, fully supported LLM training framework with broad model support
• you want turnkey training of mainstream LLMs like Llama with optimized kernels
• you need extensive documentation and enterprise support

Facets

library · maturity active

llm-training machine-learning deep-learning large-language-models deep-learning machine-learning artificial-intelligence python self-rewarding dpo spin transformers research-implementation pytorch fine-tuning lucidrains

Member repositories

For agents

markdown · JSON · MCP: product_card(name="lucidrains/self-rewarding-lm-pytorch")