Open-source benchmarking software

817 open-source libraries, tools and applications tagged benchmarking, ranked by how closely benchmarking is their primary purpose, then by adoption and maintenance health. Each entry links to a page with sources, use cases and when to choose or avoid it.

817 products, primary matches first, then adoption-weighted; health v2 shown.

Product	Health v2	Stars	Maturity
k6 Grafana k6 is an open-source load and performance testing tool with a Go-based engine and tests written in JavaScript or TypeScript. It sup…	99	31932	stable
sharkdp/hyperfine Hyperfine is a cross-platform command-line benchmarking tool written in Rust. It runs shell commands multiple times, performs statistical a…	59	28913	stable
locustio/locust Locust is an open-source load and performance testing framework where test scenarios are written in plain Python code. It supports distribu…	95	28227	active
tsenart/vegeta Vegeta is a versatile HTTP load testing tool and Go library that drills HTTP services with a constant request rate. It is usable both as a …	68	25203	stable
recommenders-team/recommenders A Python library and collection of Jupyter notebooks with best practices for building, evaluating, and operationalizing recommendation syst…	66	21919	active
aidenybai/react-scan React Scan is a drop-in tool that automatically detects and visually highlights React component performance issues, such as unnecessary re-…	66	21868	active
catchorg/Catch2 Catch2 is a modern, C++-native unit testing framework supporting TDD and BDD styles, with no external dependencies. It also provides basic …	94	21515	stable
hey hey is a small command-line HTTP load generator written in Go, originally named boom and designed as a replacement for ApacheBench (ab). It…	53	20285	stable
brendangregg/FlameGraph The original open-source FlameGraph tools by Brendan Gregg, a set of Perl scripts that convert profiler stack traces into interactive SVG f…	23	19790	stable
openai/evals OpenAI Evals is a Python framework for evaluating large language models and LLM-based systems, with an open-source registry of benchmark ev…	63	19500	active
Vitest Vitest is a next-generation JavaScript/TypeScript testing framework built on top of Vite, offering Jest-compatible APIs with fast watch mod…	94	17202	stable
wolfpld/tracy Tracy is a real-time, nanosecond-resolution, hybrid frame and sampling profiler for games and other applications. It supports CPU and GPU p…	96	16856	stable
System Informer System Informer (formerly Process Hacker) is a free, open-source, multi-purpose Windows tool for monitoring system resources, debugging sof…	86	16265	active
vibrantlabsai/ragas Ragas is a Python library for evaluating LLM applications with objective, LLM-driven metrics and systematic experiment loops. It also gener…	66	15901	active
benfred/py-spy py-spy is a sampling profiler for Python programs written in Rust. It visualizes where a running Python process spends time without restart…	79	15553	stable
Memray Memray is a memory profiler for Python that tracks memory allocations in Python code, native C/C++ extension modules, and the interpreter i…	92	15385	active
Optuna Optuna is an open-source Python hyperparameter optimization framework with a define-by-run API for dynamically constructing search spaces. …	93	14894	active
EleutherAI/lm-evaluation-harness A Python framework for few-shot evaluation of language models across hundreds of standard benchmarks (HellaSwag, MMLU, BIG-Bench, etc.). It…	91	14198	active
RL Baselines3 Zoo Stable Baselines3 is a Python library providing reliable, well-tested PyTorch implementations of reinforcement learning algorithms with a u…	86	13883	stable
plasma-umass/scalene Scalene is a high-performance CPU, GPU, and memory profiler for Python that runs much faster than many alternatives while providing detaile…	88	13514	active
ShishirPatil/gorilla Gorilla is a UC Berkeley research project for training and evaluating LLMs to make function/tool calls, including the OpenFunctions fine-tu…	52	13044	active
Taskflow Taskflow is a header-only C++ library for building task-parallel programs using task dependency graphs, with a work-stealing executor for C…	84	12188	stable
dotnet/BenchmarkDotNet BenchmarkDotNet is a powerful .NET library for benchmarking that transforms methods into benchmarks and produces reliable, statistically so…	83	11510	stable
pathwaycom/arc-task-gen A Python tool that generates original ARC-AGI-1-style puzzle tasks distribution-matched to the public evaluation set. It produces tasks in …	53	11149	active
google/dopamine Dopamine is a research framework from Google for fast prototyping of reinforcement learning algorithms, built around a small, easily readab…	52	10925	active
hatoo/oha oha is a small HTTP load generator written in Rust, inspired by rakyll/hey, that sends load to a web application and displays results in a …	93	10601	active
vwxyzjn/cleanrl CleanRL is a deep reinforcement learning library providing high-quality, single-file implementations of algorithms like PPO, DQN, DDPG, TD3…	54	10481	active
google/benchmark Google Benchmark is a C++ microbenchmarking library for timing small code snippets, with an API similar to unit test frameworks. It support…	91	10464	stable
XMRig XMRig is a high-performance, open-source, cross-platform CPU/GPU miner for RandomX, KawPow, CryptoNight, and GhostRider algorithms, plus a …	84	10137	active
mshumer/gpt-prompt-engineer A Jupyter Notebook-based tool that automatically generates, tests, and ranks LLM prompts for a given task using GPT-4, GPT-3.5-Turbo, or Cl…	47	9681	active
apache/jmeter Apache JMeter is a 100% Java open-source application for load testing and measuring the performance of web applications and other services.…	66	9552	stable
pprof pprof is a tool for visualization and analysis of profiling data stored in profile.proto format. It generates text and graphical reports (f…	77	9301	active
async-profiler/async-profiler A low-overhead sampling profiler for Java applications built on HotSpot's AsyncGetCallTrace API and Linux perf_events. It profiles CPU time…	96	9169	stable
flightlessmango/MangoHud MangoHud is a Vulkan and OpenGL overlay that displays real-time performance metrics like FPS, frametimes, CPU/GPU load, and temperatures in…	88	9128	active
HypothesisWorks/hypothesis Hypothesis is the property-based testing library for Python, generating randomized test inputs from user-described strategies and shrinking…	96	9092	stable
onsi/ginkgo Ginkgo is a mature BDD-style testing framework for Go that builds on Go's native testing package. It provides an expressive DSL for structu…	96	9058	stable
esnet/iperf iperf3 is a command-line tool for actively measuring the maximum achievable bandwidth on IP networks over TCP, UDP, and SCTP. It reports th…	90	8816	active
Jesse Jesse is an open-source Python framework for researching, backtesting, optimizing, and live-trading cryptocurrency trading strategies. It i…	77	8660	active
HKUDS/ClawWork ClawWork is an open-source benchmark and framework that turns AI assistants into 'AI coworkers' which must earn income by completing real p…	44	8560	active
autocannon A fast HTTP/1.1 benchmarking and load-testing tool written in Node.js, inspired by wrk and wrk2. It can be used from the command line or pr…	56	8520	stable
wg/wrk wrk is a modern HTTP benchmarking tool that generates significant load from a single multi-core machine using a multithreaded design with e…	32	40417	maintenance
davebcn87/pi-autoresearch A pi extension that runs autonomous optimization loops: try an idea, benchmark it, keep improvements, and revert regressions. Inspired by k…	81	8162	active
TechEmpower/FrameworkBenchmarks The TechEmpower Framework Benchmarks (TFB) project, a suite that measures and compares the performance of web application frameworks across…	10	8106	active
joerick/pyinstrument pyinstrument is a call stack profiler for Python that helps identify why code is slow. It supports profiling scripts, async code, Django ap…	91	8015	active
Scientist Scientist is a Ruby library for carefully refactoring critical code paths by running old and new implementations side by side in production…	66	7769	stable
gaogaotiantian/viztracer VizTracer is a low-overhead debugging and profiling tool that traces and visualizes Python code execution on an interactive timeline. It re…	76	7754	active
OpenMathLib/OpenBLAS OpenBLAS is an optimized open-source implementation of the BLAS (Basic Linear Algebra Subprograms) library, based on GotoBLAS2, with LAPACK…	89	7587	stable
jepsen-io/jepsen Jepsen is a Clojure library and framework for verifying the correctness of distributed systems under fault injection. It sets up systems on…	95	7528	active
krausest/js-framework-benchmark A benchmark suite comparing the rendering performance of ~186 JavaScript web frameworks by measuring DOM table operations, memory usage, an…	95	7494	active
open-compass/opencompass OpenCompass is a Python-based evaluation platform and toolkit for benchmarking large language models across 100+ datasets, supporting model…	93	7471	active
GoogleChrome/lighthouse-ci Lighthouse CI is a suite of tools for continuously running, saving, retrieving, and asserting against Lighthouse audit results in a CI pipe…	79	7114	active
TheTom/turboquant_plus TurboQuant+ is a Python reference implementation of the TurboQuant KV cache compression method (ICLR 2026), using PolarQuant codebooks and …	54	7036	active
ai/size-limit Size Limit is a performance budget tool for JavaScript that calculates the real cost (size and execution time) of your app or library and f…	98	6971	stable
Gatling Gatling is an open-source, code-first load testing tool and performance engineering platform. It uses a fully asynchronous, non-blocking ar…	87	6955	stable
doctest/doctest doctest is a single-header C++ testing framework for unit tests and TDD, supporting C++11 through C++23. It is designed to be the fastest f…	87	6889	stable
codesenberg/bombardier Bombardier is a fast cross-platform HTTP(S) benchmarking tool written in Go, built on fasthttp with an optional net/http client for HTTP/2.…	78	6841	active
NicolasHug/Surprise Surprise is a Python scikit for building and analyzing recommender systems that work with explicit rating data. It provides ready-to-use pr…	67	6817	stable
akopytov/sysbench sysbench is a scriptable, multi-threaded benchmark tool built on LuaJIT, most commonly used for OLTP database benchmarks against MySQL and …	24	6802	stable
jlfwong/speedscope speedscope is a fast, interactive web-based viewer for performance profiles that imports data from many profilers across languages like Jav…	67	6771	stable
masonr/yet-another-bench-script YABS is a self-contained bash script that benchmarks Linux servers by running fio for disk I/O, iperf3 for network throughput, and Geekbenc…	96	6699	active
google/perfetto Perfetto is an open-source suite of SDKs, daemons, and tools for production-grade system profiling, app tracing, and trace analysis on clie…	93	6647	stable
DEAP DEAP is a Python framework for evolutionary computation that supports genetic algorithms, genetic programming, evolution strategies, and mu…	63	6450	stable
axboe/fio Fio is a flexible I/O benchmarking and workload simulation tool that spawns threads or processes performing user-defined I/O patterns via j…	89	6391	stable
JoeDog/siege Siege is an open-source HTTP load testing and benchmarking utility written in C. It simulates many concurrent users hitting one or many URL…	85	6216	active
alexcasalboni/aws-lambda-power-tuning AWS Lambda Power Tuning is an open-source tool built on AWS Step Functions that benchmarks Lambda functions across different memory configu…	75	6068	active
flamegraph-rs/flamegraph A Rust-powered flamegraph generator and cargo subcommand built on the Inferno library, profiling arbitrary binaries without Perl or pipes. …	96	6047	active
AgentOps-AI/agentops AgentOps is a Python SDK and observability platform for monitoring, debugging, and evaluating AI agents and LLM applications. It auto-instr…	67	5886	active
microsoft/ethr Ethr is a cross-platform network performance measurement tool written in Go that measures bandwidth, connections/s, packets/s, latency, los…	60	5868	active
baidu/dperf dperf is a high-performance network traffic generator and load testing tool built on DPDK, capable of tens of millions of HTTP connections …	62	5644	active
lm-sys/RouteLLM RouteLLM is a Python framework for serving and evaluating LLM routers that route queries between strong and weak models to cut costs. It pr…	25	5626	active
google/bloaty Bloaty is a size profiler for binaries that shows what is taking up space inside an executable. It uses custom ELF, DWARF, and Mach-O parse…	66	5563	active
Netflix/vmaf VMAF is Netflix's Emmy-winning perceptual video quality assessment algorithm, provided as a standalone C library (libvmaf) with a wrapping …	93	5516	stable
teddysun/across A collection of shell scripts for Linux VPS/server administration, including WireGuard VPN setup, TCP BBR kernel installation, KMS server d…	62	5376	active
KDAB/hotspot Hotspot is a standalone Qt-based GUI for analyzing Linux perf performance data, providing flame graphs, timelines, caller/callee views, and…	76	5186	active
amanusk/s-tui s-tui is a terminal-based UI that graphically monitors CPU temperature, frequency, utilization, and power, and highlights thermal throttlin…	86	5100	active
sitespeed.io sitespeed.io is an open-source web performance testing and monitoring tool that drives real browsers (Chrome, Firefox, Edge, Safari) to col…	92	5037	active
jazzband/django-silk django-silk (Silk) is a live profiling and inspection tool for the Django framework. It intercepts and stores HTTP requests and database qu…	98	4995	active
Nixtla/statsforecast StatsForecast is a Python library providing fast, scalable implementations of statistical and econometric univariate time series forecastin…	87	4938	stable
pytorch/ignite PyTorch-Ignite is a high-level library for training and evaluating neural networks in PyTorch flexibly and transparently. It provides an En…	84	4787	stable
microsoft/perfview PerfView is a free Windows-based performance-analysis tool for isolating CPU and memory performance issues, with special support for .NET a…	95	4766	active
TCPCopy TCPCopy is a TCP stream replay tool that captures live production traffic and replicates it against a test server. It enables realistic str…	31	4686	stable
nolanlawson/fuite fuite is a CLI tool that finds memory leaks in web apps by launching Chrome via Puppeteer, running scenarios (like clicking links and press…	45	4621	active
thirdlayerinc/autoagent AutoAgent is a Python-based meta-agent harness that autonomously builds and iterates on AI agent harnesses. It edits an agent's system prom…	45	4575	active
plasma-umass/coz Coz is a causal profiler for native code written in C/C++ and Rust. It runs performance experiments to predict how optimizing specific line…	79	4552	active
six-ddc/plow Plow is a high-performance HTTP(S) benchmarking tool written in Go, built on fasthttp for maximum throughput. It reports real-time latency …	67	4514	active
salesforce/Merlion Merlion is a Python library from Salesforce providing an end-to-end machine learning framework for time series intelligence, covering forec…	10	4486	active
EvolvingLMMs-Lab/lmms-eval lmms-eval is a unified Python framework for evaluating large multimodal models across text, image, video, and audio tasks, with 100+ benchm…	87	4453	active
mstange/samply Samply is a command-line sampling CPU profiler for macOS, Linux, and Windows that records profiles of arbitrary commands and displays them …	66	4426	active
open-compass/VLMEvalKit VLMEvalKit is an open-source Python toolkit for evaluating large vision-language models (LMMs/LVLMs) across 80+ benchmarks with support for…	61	4412	active
link1st/go-stress-testing A stress/load testing tool written in Go that simulates concurrent users with goroutines to measure QPS and connection capacity. It support…	72	4408	active
DistrictDataLabs/yellowbrick Yellowbrick is a Python library of visual diagnostic tools called Visualizers that extend the scikit-learn API using matplotlib. It helps u…	23	4405	active
MichaelGrupp/evo evo is a Python package with CLI executables and a small library for evaluating, comparing, and plotting trajectory output from odometry an…	74	4318	stable
facebookresearch/nevergrad Nevergrad is a Python toolbox for gradient-free (derivative-free) optimization, providing many optimization algorithms such as NGOpt and di…	79	4215	active
DLRM A PyTorch-based implementation of the Deep Learning Recommendation Model (DLRM) from Facebook Research, which processes dense and sparse fe…	57	4074	stable
nakabonne/ali ali is a Go-based HTTP load testing CLI tool that generates HTTP traffic against a target URL and plots performance metrics in real time wi…	54	3940	active
BrentOzarULTD/SQL-Server-First-Responder-Kit A collection of T-SQL stored procedures (sp_Blitz, sp_BlitzCache, sp_BlitzFirst, sp_BlitzIndex, and more) for Microsoft SQL Server health c…	88	3914	active
MiniProfiler/rack-mini-profiler Rack middleware that displays a speed badge on every HTML page with detailed performance profiling for Ruby web apps. It provides database …	70	3904	active
openai/baselines OpenAI Baselines is a collection of high-quality reference implementations of reinforcement learning algorithms such as DQN, PPO, and ACKTR…	32	16764	maintenance
roboflow/trackers A Python library of clean-room, Apache 2.0 implementations of multi-object tracking algorithms including SORT, ByteTrack, OC-SORT, BoT-SORT…	85	3850	active
mseitzer/pytorch-fid A PyTorch port of the official TensorFlow implementation of the Fréchet Inception Distance (FID), a metric for measuring similarity between…	23	3847	stable