openai/lm-human-preferences

Health v2 · maintenance only

Flags: no_releases archived

Adoption not part of the score

1391 stars · 171 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

OpenAI's research code for the paper 'Fine-Tuning Language Models from Human Preferences', implementing reward model training from human labels and RL fine-tuning of GPT-2 language models. It is an archived repository provided as-is with no updates expected.

Use cases

• reproduce the fine-tuning from human preferences paper
• train a reward model from human preference labels
• fine-tune GPT-2 with reinforcement learning from human feedback
• study early RLHF implementations
• download released human preference labels for experiments

When to choose

• you want to study or reproduce the original 2019 RLHF research code
• you need the exact reward modeling and PPO fine-tuning setup from the paper
• you want access to the released human preference label datasets

When to avoid

• you need a modern, maintained RLHF library (use trl or similar)
• you need to fine-tune models larger than small GPT-2
• you need up-to-date TensorFlow or GPU support
• you want production-ready training pipelines

Facets

library · maturity abandoned

machine-learning llm-training reinforcement-learning large-language-models machine-learning reinforcement-learning python rlhf reward-model human-preferences gpt-2 research-code tensorflow archived research linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="openai/lm-human-preferences")