vwxyzjn/cleanrl

Health v2 · maintenance only

Flags: no_license

Adoption not part of the score

10481 stars · 1173 forks observed · 2026-10-02

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

CleanRL is a deep reinforcement learning library providing high-quality, single-file implementations of algorithms like PPO, DQN, DDPG, TD3, and SAC. Each standalone file contains all details of an algorithm variant, making it ideal for reading, benchmarking, and research prototyping.

Use cases

• understand every implementation detail of a deep RL algorithm like PPO
• prototype custom RL algorithm variants not supported by modular libraries
• benchmark RL algorithms across Atari and other environments
• run thousands of RL experiments in the cloud with AWS Batch
• learn deep reinforcement learning from readable reference code
• reproduce RL research results with seeded, logged experiments

When to choose

• you want readable, self-contained reference implementations of RL algorithms
• you need to modify or extend algorithm internals for research
• you want benchmarked implementations with TensorBoard and W&B logging
• you prefer copy-paste single files over importing a modular library

When to avoid

• you want a modular library to import and compose RL components in your own code
• you need offline RL algorithms (use CORL instead)
• you want production-ready, feature-complete RL tooling with minimal code duplication
• you need a stable API surface, since files are meant to be copied and edited

Facets

library · maturity active

machine-learning reinforcement-learning benchmarking sdk reinforcement-learning machine-learning deep-learning tutorials python cloud cross-platform deep-reinforcement-learning pytorch ppo dqn sac single-file-implementations gymnasium atari wandb reference-implementations research gpu docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="vwxyzjn/cleanrl")