# vwxyzjn/cleanrl

High-quality single file implementation of Deep Reinforcement Learning algorithms with research-friendly features (PPO, DQN, C51, DDPG, TD3, SAC, PPG)

Repository: https://github.com/vwxyzjn/cleanrl
Canonical: https://ross.abutalabs.com/products/cleanrl
Homepage: http://docs.cleanrl.dev
Language: Python
License: NOASSERTION
License Family: other
Topics: wandb, reinforcement-learning, pytorch, python, gym, machine-learning, deep-reinforcement-learning, deep-learning, atari, ale, a2c, proximal-policy-optimization, ppo, advantage-actor-critic, actor-critic, phasic-policy-gradient
Last push: 2026-04-20T10:57:15+00:00

## Health v2 (maintenance only)
Score: 58/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 78, release rhythm 8, longevity 100
- inputs: {"age_days": 2644, "days_push": 135, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 10326, forks 1159 (observed 2026-08-28T04:10:41.654910+00:00)

## What it is
CleanRL is a deep reinforcement learning library providing high-quality, single-file implementations of algorithms like PPO, DQN, DDPG, TD3, and SAC. Each standalone file contains all details of an algorithm variant, making it ideal for reading, benchmarking, and research prototyping.

## Use cases
- understand every implementation detail of a deep RL algorithm like PPO
- prototype custom RL algorithm variants not supported by modular libraries
- benchmark RL algorithms across Atari and other environments
- run thousands of RL experiments in the cloud with AWS Batch
- learn deep reinforcement learning from readable reference code
- reproduce RL research results with seeded, logged experiments

## When to choose
- you want readable, self-contained reference implementations of RL algorithms
- you need to modify or extend algorithm internals for research
- you want benchmarked implementations with TensorBoard and W&B logging
- you prefer copy-paste single files over importing a modular library

## When to avoid
- you want a modular library to import and compose RL components in your own code
- you need offline RL algorithms (use CORL instead)
- you want production-ready, feature-complete RL tooling with minimal code duplication
- you need a stable API surface, since files are meant to be copied and edited

## Facets
- artifact type: library
- maturity: active
- function: machine-learning, reinforcement-learning, benchmarking, sdk
- domain: reinforcement-learning, machine-learning, deep-learning, tutorials
- platform: python, cloud, cross-platform
- tags: deep-reinforcement-learning, pytorch, ppo, dqn, sac, single-file-implementations, gymnasium, atari, wandb, reference-implementations, research, gpu, docker

## Member repositories
- vwxyzjn/cleanrl (main) score 58

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:10:41.654910+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T17:19:09.273718+00:00, confidence not recorded.
  - readme: https://github.com/vwxyzjn/cleanrl (fetched 2026-08-28T04:10:41.654910+00:00, sha 064097c19072)
  - homepage: http://docs.cleanrl.dev (fetched 2026-08-29T08:18:29.817269+00:00, sha eb3be7b71e53)
  - site_page: https://docs.cleanrl.dev/get-started/installation (fetched 2026-08-29T08:18:29.826792+00:00, sha c1a7cf838dcc)
  - site_page: https://docs.cleanrl.dev/cloud/installation (fetched 2026-08-29T08:18:29.828629+00:00, sha d1152ee0250e)
- Data as of 2026-08-30T08:39:29.467469+00:00.