Open-source benchmarking software
817 open-source libraries, tools and applications tagged benchmarking, ranked by how closely benchmarking is their primary purpose, then by adoption and maintenance health. Each entry links to a page with sources, use cases and when to choose or avoid it.
817 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| facebook/fatal Fatal (Facebook Template Library) is a header-only C++14 library for fast software prototyping, built heavily on template meta-programming … | 10 | 1018 | maintenance |
| httperf/httperf httperf is a command-line HTTP load generator for measuring web server performance. It supports HTTP/1.1 and SSL and is designed to generat… | 32 | 1016 | maintenance |
| nnethercote/dhat-rs A Rust crate providing heap profiling and ad hoc profiling capabilities similar to Valgrind's DHAT, without requiring external tools. It al… | 23 | 1005 | maintenance |
| maintell/webBenchmark A Go-based command-line HTTP benchmark tool that floods a target URL with concurrent requests to test server bandwidth and performance. It … | 23 | 1004 | maintenance |
| jennyzzt/dgm Darwin Gödel Machine (DGM) is a research implementation of a self-improving AI agent that iteratively rewrites its own codebase and validat… | 32 | 2424 | experimental |
| FastVM/minivm MiniVM is a small but optimizing virtual machine and runtime written in C11, featuring a just-in-time compiler built on Cuik's TB backend. … | 30 | 1667 | experimental |
| Orion Orion is a Python machine learning library for unsupervised time series anomaly detection, offering verified ML pipelines that flag rare pa… | 73 | 1370 | experimental |
| erikbern/ann-benchmarks A benchmarking environment for approximate nearest neighbor (ANN) search algorithms, with pre-generated HDF5 datasets and Docker containers… | 70 | 5735 | abandoned |
| uber-archive/go-torch go-torch is a command-line tool that stochastically profiles Go programs by collecting stack traces via pprof and rendering them as flame g… | 10 | 3932 | abandoned |
| facebookresearch/ReAgent ReAgent is an open-source end-to-end platform for applied reinforcement learning developed at Facebook, built in Python with PyTorch for tr… | 74 | 3722 | abandoned |
| rkern/line_profiler line_profiler is a Python module for line-by-line profiling of functions, and kernprof is a companion script for running it or the standard… | 10 | 3582 | abandoned |
| rcrowley/go-metrics A Go port of Coda Hale's Dropwizard Metrics library providing counters, gauges, histograms, meters, and timers for application instrumentat… | 10 | 3462 | abandoned |
| codemix/fast.js fast.js is a JavaScript library providing faster user-land reimplementations of common native built-in functions like forEach, map, and red… | 32 | 3381 | abandoned |
| rll/rllab rllab is a Python framework for developing and evaluating reinforcement learning algorithms, with implementations of algorithms like TRPO, … | 32 | 3079 | abandoned |
| uber-archive/pyflame Pyflame is a high-performance sampling profiler for Python implemented in C++ that uses Linux ptrace to capture call stacks without instrum… | 10 | 2979 | abandoned |
| what-studio/profiling An interactive continuous Python profiler with a TUI viewer, supporting both deterministic and statistical profiling, including live and re… | 10 | 2933 | abandoned |
| scrtlabs/catalyst Catalyst is a Python algorithmic trading library for crypto-assets, built as an extension of Quantopian's Zipline. It supports backtesting … | 10 | 2559 | abandoned |
| facebookarchive/BOLT BOLT (Binary Optimization and Layout Tool) is a Linux command-line utility that optimizes the performance of native binaries by rearranging… | 10 | 2532 | abandoned |
| brookshi/Hitchhiker Hitchhiker is a self-hosted, web-based RESTful API testing tool built with Node.js and TypeScript. It supports team collaboration, schedule… | 23 | 2206 | abandoned |
| seiginonakama/BlockCanaryEx BlockCanaryEx is an Android library, based on BlockCanary, that detects UI thread blocks and identifies the heavy methods causing them. It … | 32 | 2172 | abandoned |
| reactopt/reactopt Reactopt is a CLI tool that audits React applications for unnecessary component re-rendering by launching the app in a browser via Puppetee… | 32 | 1971 | abandoned |
| friendlyrobotnyc/TinyDancer An Android library that displays a real-time FPS overlay using the Choreographer, with color-coded indicators based on the percentage of ti… | 10 | 1890 | abandoned |
| goadapp/goad Goad is a highly distributed load testing tool written in Go that runs HTTP load tests from AWS Lambda across multiple regions simultaneous… | 10 | 1866 | abandoned |
| speedracer/speedracer Speedracer is a performance runner, like a test runner but for performance, that executes 'race' scripts in headless Chrome and collects de… | 23 | 1848 | abandoned |
| stagemonitor/stagemonitor Stagemonitor is an open-source application performance monitoring agent for Java server applications that integrates with Elasticsearch, Gr… | 10 | 1694 | abandoned |
| xiaojiaqi/10billionhongbaos A Go-based stress-testing project that simulates the 2015 WeChat red envelope scenario, supporting one million concurrent users on a single… | 32 | 1689 | abandoned |
| uber-archive/cpustat cpustat is a Linux command-line tool that samples CPU utilization and saturation for every process at high frequency and summarizes the sam… | 10 | 1649 | abandoned |
| tideways/php-xhprof-extension A PHP C extension (tideways_xhprof) providing hierarchical, XHProf-compatible callgraph profiling for PHP 7+ with low overhead. The project… | 10 | 1621 | abandoned |
| samccone/drool Drool is a Node.js automation layer built on chromedriver/Selenium WebDriver that measures whether repeated 'clean' user actions cause DOM … | 32 | 1470 | abandoned |
| jonathanpenn/ui-auto-monkey UI AutoMonkey is a JavaScript stress testing script for iOS apps that runs in Apple's UI Automation and Instruments tools, bombarding apps … | 32 | 1458 | abandoned |
| joedicastro/vps-comparison A Python and Ansible-based tool that runs automated benchmark tests across VPS providers and publishes reproducible comparison results. It … | 32 | 1433 | abandoned |
| twitter-archive/iago Iago is a load generator from Twitter for producing realistic traffic against services, written in Scala on the JVM. Users write Scala or J… | 10 | 1333 | abandoned |
| paulirish/pwmetrics PWMetrics is a CLI tool and Node.js library that gathers progressive web performance metrics (like First Contentful Paint and Largest Conte… | 23 | 1238 | abandoned |
| humbug/humbug Humbug is a mutation testing framework for PHP that measures the real effectiveness of test suites by injecting small defects and checking … | 10 | 1125 | abandoned |
| node-inspector/v8-profiler Node.js native bindings (C++) to the V8 profiler, exposing CPU profiling and heap snapshot APIs programmatically. It integrates with node-i… | 23 | 1122 | abandoned |
| lafikl/perfBar PerfBar is a lightweight JavaScript library for collecting and displaying website performance metrics directly in the browser, with support… | 23 | 1089 | abandoned |
| SamSaffron/MiniProfiler MiniProfiler is a simple but effective mini-profiler for ASP.NET MVC, ASP.NET, and Ruby, providing ADO.NET query profiling and lightweight … | 32 | 1043 | abandoned |
| GPUOpen-Archive/CodeXL CodeXL is AMD's open-source GPU tool suite offering GPU debugging, profiling, and static kernel/shader analysis for OpenCL, OpenGL, Vulkan,… | 10 | 1011 | abandoned |
| Qlib Qlib is an AI-oriented quantitative investment platform from Microsoft that supports the full quant research workflow, from data processing… | 66 | 48694 | active |
| ray-project/ray Ray is a unified open-source framework for scaling AI and Python applications, consisting of a core distributed runtime (tasks, actors, obj… | 99 | 44057 | stable |
| HKUDS/Vibe-Trading Vibe-Trading is an open-source AI finance research agent that turns natural-language prompts into runnable market research, backtests, and … | 82 | 35591 | active |
| derailed/k9s K9s is a terminal-based UI for navigating, observing, and managing Kubernetes clusters in real time. It supports standard resources and CRD… | 93 | 34789 | active |
| open-mmlab/mmdetection MMDetection is OpenMMLab's PyTorch-based toolbox and benchmark for object detection and instance/panoptic segmentation. It provides a large… | 23 | 32955 | stable |
| ifixai-ai/iFixAi iFixAi is an open-source CLI diagnostic tool that independently audits AI agents for misalignment, hallucination, manipulation, deception, … | 82 | 28773 | active |
| MLflow MLflow is an open source AI engineering platform for agents, LLMs, and traditional ML models, providing experiment tracking, model registry… | 100 | 28441 | stable |
| TOON TOON (Token-Oriented Object Notation) is a compact, human-readable encoding of the JSON data model designed to reduce token usage in LLM pr… | 78 | 25440 | active |
| Dao-AILab/flash-attention Official implementation of FlashAttention, FlashAttention-2, -3, and -4: fast and memory-efficient exact attention kernels for GPUs. It acc… | 92 | 25116 | active |
| Apache SkyWalking Apache SkyWalking is an open‑source Application Performance Monitoring (APM) platform designed for cloud‑native, microservice, and containe… | 88 | 24968 | active |
| simdjson A C++ library that parses JSON at gigabytes-per-second speeds using SIMD instructions and microparallel algorithms, with runtime CPU-tailor… | 100 | 24432 | stable |
| HikariCP HikariCP is a lightweight, high-performance JDBC connection pool for Java applications. It is a production-ready, 'zero-overhead' library w… | 68 | 21240 | stable |
| confident-ai/deepeval DeepEval is an open-source Python framework for evaluating LLM applications with pytest-style unit tests and 50+ research-backed metrics in… | 95 | 18733 | active |
| infinitered/reactotron Reactotron is a free, open-source desktop application for debugging React JS and React Native apps, letting developers inspect state, API r… | 92 | 15590 | active |
| apache/tvm Apache TVM is an open machine learning compiler framework that takes pre-trained models and compiles them into optimized, deployable module… | 95 | 13840 | active |
| deepseek-ai/FlashMLA FlashMLA is DeepSeek's library of optimized CUDA attention kernels implementing Multi-head Latent Attention (MLA), including dense and toke… | 64 | 13063 | active |
| Arize Phoenix Arize Phoenix is an open-source AI observability and evaluation platform for tracing, debugging, and evaluating LLM applications. It suppor… | 95 | 11875 | active |
| tensorzero/tensorzero TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation behi… | 10 | 11716 | active |
| Weights & Biases Weights & Biases (wandb) is a Python SDK and platform for tracking, visualizing, and managing machine learning experiments, including metri… | 100 | 11303 | active |
| sktime sktime is a unified Python framework for machine learning with time series, offering 500+ models behind a single scikit-learn-compatible AP… | 98 | 10062 | stable |
| major/MySQLTuner-perl MySQLTuner is a Perl script that audits a running MySQL, MariaDB, or Percona Server instance and reports configuration variables, status me… | 97 | 9479 | active |
| Artillery Artillery is an open-source load testing platform and CLI for stress-testing HTTP APIs, GraphQL, WebSocket, Socket.IO, gRPC, and browser ap… | 90 | 9091 | active |
| kernc/backtesting.py Backtesting.py is a lightweight Python framework for backtesting trading strategies against historical candlestick data. It provides a smal… | 72 | 9067 | active |
| karatelabs/karate Karate is an open-source Java-based test automation framework that unifies API testing (REST, GraphQL, SOAP, WebSocket), UI browser automat… | 99 | 8985 | active |
| deepseek-ai/DeepGEMM DeepGEMM is a high-performance CUDA BLAS kernel library for NVIDIA tensor cores, providing FP8, FP4, and BF16 GEMMs plus fused MoE and othe… | 82 | 8937 | active |
| elder-plinius/OBLITERATUS OBLITERATUS is a Python toolkit for abliteration — identifying and removing refusal behaviors from large language models by locating and in… | 79 | 8861 | active |
| boyter/scc scc is a very fast, accurate lines-of-code counter written in pure Go that also computes code complexity estimates, COCOMO/LOCOMO effort es… | 88 | 8807 | active |
| tile-ai/tilelang TileLang is a Pythonic domain-specific language and compiler (built on TVM) for writing high-performance GPU/CPU/accelerator kernels such a… | 90 | 8658 | active |
| Anteon Anteon (formerly Ddosify) is an open-source, eBPF-based Kubernetes monitoring and performance testing platform. It automatically generates … | 51 | 8517 | active |
| Dynamo NVIDIA Dynamo is an open-source, datacenter-scale distributed inference serving framework that orchestrates inference engines like vLLM, SG… | 88 | 8345 | active |
| open-mmlab/mmpose MMPose is an open-source pose estimation toolbox and benchmark built on PyTorch as part of the OpenMMLab ecosystem. It provides implementat… | 35 | 7921 | active |
| H2O H2O-3 is an open-source, distributed, in-memory machine learning platform implementing algorithms such as GLM, GBM/XGBoost, Random Forest, … | 75 | 7509 | stable |
| deepseek-ai/DeepSpec DeepSpec is a full-stack Python codebase from DeepSeek for training and evaluating draft models used in speculative decoding of large langu… | 51 | 7207 | active |
| flashinfer-ai/flashinfer FlashInfer is a GPU kernel library and kernel generator for LLM inference, providing unified APIs for attention, GEMM, and MoE operations w… | 92 | 6728 | active |
| simplescaling/s1 s1 is an open-source research project implementing simple test-time scaling for large language models, including the s1K dataset of 1,000 c… | 30 | 6675 | active |
| Harbor Harbor is a Python framework from the creators of Terminal-Bench for evaluating and optimizing AI agents and language models in containeriz… | 86 | 6309 | active |
| Helicone Helicone is an open-source LLM observability platform and AI gateway that logs, monitors, and analyzes LLM API requests with a one-line int… | 78 | 6224 | active |
| open-edge-platform/anomalib Anomalib is a Python deep learning library for anomaly detection, offering state-of-the-art unsupervised algorithms for detecting and local… | 96 | 6196 | active |
| microsoft/fara Fara1.5 is a family of open-weight computer use agent (CUA) models from Microsoft Research, released at 4B, 9B, and 27B scales and built on… | 56 | 6186 | active |
| alchaincyf/darwin-skill Darwin.skill is an agent skill for Claude Code that autonomously optimizes other SKILL.md agent skills through an evaluate-improve-test-kee… | 58 | 6170 | active |
| Stdlib stdlib is a standard library for JavaScript and TypeScript with an emphasis on numerical and scientific computing, providing math, statisti… | 93 | 6000 | active |
| Clockwork Clockwork is a PHP development tool that collects runtime data from your application - request data, performance metrics, logs, database qu… | 64 | 5948 | active |
| Giskard-AI/giskard-oss Giskard is an open-source Python library for testing, evaluating, and red-teaming LLM-based agents and RAG systems. Its v3 rewrite provides… | 95 | 5910 | active |
| rllm-org/rllm rLLM is an open-source Python framework for training language agents with reinforcement learning, supporting any agent harness (Claude Code… | 75 | 5862 | active |
| pytorch/torchtitan torchtitan is a PyTorch-native platform for large-scale training of generative AI models, offering a clean-room implementation of PyTorch's… | 81 | 5830 | active |
| AidLux (AidLearning) AidLux (originally AidLearning) is an AIoT development platform that runs a native Ubuntu Linux environment with GUI, deep learning tooling… | 63 | 5812 | active |
| google-deepmind/open_spiel OpenSpiel is a collection of game environments and reinforcement learning/search/planning algorithms for research in general RL and multiag… | 93 | 5537 | active |
| ProjectPhysX/FluidX3D FluidX3D is a high-performance lattice Boltzmann method computational fluid dynamics (CFD) application written in C++/OpenCL. It runs on al… | 90 | 5290 | active |
| open-mmlab/mmaction2 MMAction2 is OpenMMLab's PyTorch-based toolbox and benchmark for video understanding, covering action recognition, temporal action localiza… | 52 | 5163 | active |
| emqx/MQTTX MQTTX is an all-in-one, cross-platform MQTT 5.0 client toolbox from EMQ, available as a desktop GUI app, CLI, and web application. It provi… | 90 | 5073 | active |
| facebook/memlab MemLab is an end-to-end testing and analysis framework from Meta for detecting JavaScript memory leaks and optimization opportunities. It d… | 77 | 5068 | active |
| nghttp2/nghttp2 nghttp2 is a mature C library implementing the HTTP/2 framing layer and HPACK header compression, plus client, server, proxy, and load-test… | 89 | 5053 | stable |
| LangWatch LangWatch is an open-source LLMOps platform for LLM evaluation, AI agent testing, tracing, and production observability. It combines agent … | 91 | 4983 | active |
| Zep A repository of example code, framework integration packages, ingestion tooling, and memory benchmarks for Zep Cloud, a managed agent memor… | 97 | 4959 | active |
| KaiyangZhou/deep-person-reid Torchreid is a PyTorch library for deep-learning person re-identification, supporting both image and video reid with end-to-end training an… | 47 | 4926 | stable |
| PrimeIntellect-ai/verifiers verifiers is a Python library for creating environments to train and evaluate large language models with reinforcement learning. It integra… | 89 | 4712 | active |
| TencentCloudADP/youtu-agent Youtu-Agent is a Python framework for building, running, and evaluating autonomous LLM agents, supporting ReAct-style single agents and pla… | 52 | 4622 | active |
| FareedKhan-dev/all-agentic-architectures A Python library packaging 35 production-grade agentic AI architectures (Reflexion, LATS, GraphRAG, MemGPT, Voyager, etc.) as runnable Arch… | 57 | 4600 | active |
| RecBole RecBole is a unified, comprehensive and efficient recommendation library built on Python and PyTorch for reproducing and developing recomme… | 23 | 4569 | stable |
| OAID/Tengine Tengine is a lightweight, high-performance, modular deep learning inference engine developed by OPEN AI LAB for embedded and edge devices. … | 24 | 4531 | active |
| mosaicml/llm-foundry LLM Foundry is a PyTorch-based codebase for training, finetuning, evaluating, and deploying large language models from 125M to 70B+ paramet… | 62 | 4449 | active |
| microsoft/FLAML FLAML is a lightweight Python library for automated machine learning (AutoML) and hyperparameter tuning. It efficiently finds quality model… | 94 | 4409 | active |