lucidrains/PaLM-rlhf-pytorch

Health v2 · maintenance only

Adoption not part of the score

7861 stars · 670 forks observed · 2026-10-21

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

A PyTorch library implementing Reinforcement Learning from Human Feedback (RLHF) on top of the PaLM transformer architecture, aiming to replicate ChatGPT-style training in the open. It provides the training scaffolding (reward model, PPO) but ships no pretrained model weights.

Use cases

• train a language model with reinforcement learning from human feedback
• implement RLHF on a PaLM-style transformer in pytorch
• build an open-source ChatGPT-like model from scratch
• study how reward models and PPO are implemented for LLMs
• experiment with human preference fine-tuning of transformers

When to choose

• you want readable reference code for RLHF components like reward modeling and PPO
• you are doing research on preference-based fine-tuning and want a customizable starting point
• you already have compute and data and need the RLHF training scaffolding

When to avoid

• you need a ready-to-use chatbot or pretrained model for inference
• you want a production-grade, well-supported RLHF framework
• you prefer simpler preference optimization methods like DPO over PPO-based RLHF

Facets

library · maturity experimental

llm-training reinforcement-learning machine-learning deep-learning large-language-models reinforcement-learning deep-learning artificial-intelligence python rlhf palm reward-model ppo transformers research-code open-source-chatgpt

Member repositories

For agents

markdown · JSON · MCP: product_card(name="lucidrains/PaLM-rlhf-pytorch")