function: benchmarking
824 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| marlbenchmark/on-policy The official PyTorch implementation of Multi-Agent PPO (MAPPO), an on-policy reinforcement learning algorithm for cooperative multi-agent s… | 32 | 2085 | active |
| enarjord/passivbot Passivbot is a cryptocurrency trading bot written in Python and Rust that automatically places and cancels limit orders on perpetual future… | 98 | 2078 | active |
| LloydAsp/NodeQuality A shell-based VPS testing tool that runs benchmark scripts (Yabs, IP quality, network quality) inside a temporary chroot sandbox (BenchOS) … | 52 | 2074 | active |
| linkedin/kafka-monitor Xinfra Monitor (formerly Kafka Monitor) is a Java framework for running long-running Kafka system tests and monitoring Kafka cluster health… | 27 | 2064 | active |
| test-prof/test-prof TestProf is a Ruby gem providing a toolbox of profilers and helpers for analyzing and speeding up test suites, with integrations for ruby-p… | 77 | 2046 | active |
| Cloud-CV/EvalAI EvalAI is an open-source platform for evaluating and comparing machine learning and AI algorithms at scale. It provides a central leaderboa… | 67 | 2039 | active |
| ruby-prof/ruby-prof ruby-prof is a fast code profiler for Ruby that measures method call times and generates call graphs and reports. It helps developers ident… | 72 | 2034 | active |
| andrewrk/poop A Linux command-line benchmarking tool that compares the performance of multiple commands using perf_event_open hardware counters, with a c… | 56 | 2031 | active |
| elastic/rally Rally is a macrobenchmarking framework for Elasticsearch, developed in Python by Elastic. It automates cluster setup and teardown, benchmar… | 83 | 2029 | active |
| phpbench/phpbench PHPBench is a benchmarking framework for PHP, analogous to PHPUnit but for measuring performance rather than correctness. It runs code repe… | 90 | 2018 | active |
| leinlin/Miku-LuaProfiler A Lua profiler plugin for Unity that instruments xLua-based Lua code to report per-function CPU time and GC memory usage. It works in the U… | 50 | 2015 | active |
| tatsu-lab/alpaca_eval AlpacaEval is an LLM-based automatic evaluation framework for instruction-following language models, producing win rates against a GPT-4 ba… | 36 | 2012 | active |
| GoogleCloudPlatform/PerfKitBenchmarker PerfKit Benchmarker is a Python tool that automates running a canonical set of benchmarks across cloud providers like GCP, AWS, and Azure. … | 67 | 2011 | active |
| clementchadebec/benchmark_VAE Pythae is a PyTorch library that unifies implementations of many Variational Autoencoder (VAE) variants under a common interface, enabling … | 23 | 1995 | active |
| Hyperopt Hyperopt is a Python library for distributed asynchronous hyperparameter optimization over search spaces with real-valued, discrete, and co… | 86 | 7592 | maintenance |
| luigirizzo/netmap Netmap is a framework for extremely fast packet I/O from userspace, implemented as a kernel module for FreeBSD, Linux, and Windows. It incl… | 66 | 1974 | stable |
| twitter/pelikan Pelikan is Twitter's framework for building high-performance cache services, written in Rust with a legacy C codebase. It includes products… | 23 | 1957 | active |
| RRZE-HPC/likwid LIKWID is a command-line tool suite and library for performance-oriented programmers on Linux, supporting Intel, AMD, ARMv8, and POWER9 CPU… | 90 | 1934 | active |
| charkost/prosopite Prosopite is a Ruby gem that auto-detects N+1 queries in Rails applications with zero false positives or false negatives. It monitors SQL q… | 78 | 1906 | active |
| mlco2/codecarbon CodeCarbon is a lightweight Python library and CLI that estimates and tracks CO2 emissions from local computing hardware (CPU, GPU, RAM) by… | 98 | 1905 | active |
| max0x7ba/atomic_queue A C++14 header-only library providing multiple-producer multiple-consumer lock-free queues built on circular buffers and std::atomic. It is… | 77 | 1893 | stable |
| spiritLHLS/ecs A shell-based 'Fusion Monster' VPS server testing script that benchmarks CPU, disk, network speed, and IP quality across many Linux distrib… | 77 | 7149 | maintenance |
| ar51an/iperf3-win-builds A repository providing pre-compiled iperf3 Windows 64-bit executables, built via Cygwin without modifying the original source. It fills the… | 73 | 1887 | active |
| kyuz0/amd-strix-halo-toolboxes A collection of pre-built Toolbx/Docker containers ('toolboxes') for running llama.cpp LLM inference on AMD Ryzen AI Max 'Strix Halo' integ… | 62 | 1881 | active |
| ashvardanian/NumKong NumKong is a mixed-precision numeric and linear-algebra library with ~2,000 SIMD-accelerated kernels for dot products, distances, matrix mu… | 93 | 1877 | active |
| linkedin/greykite Greykite is a Python time series forecasting library from LinkedIn built around its Silverkite algorithm, with support for trend, seasonali… | 26 | 1855 | active |
| hcoles/pitest Pitest (PIT) is a state-of-the-art mutation testing system for Java and the JVM. It mutates bytecode to verify the effectiveness of unit te… | 98 | 1851 | active |
| linyimin0812/spring-startup-analyzer A Java agent/library that instruments Spring and Spring Boot applications to collect startup data and generate an interactive HTML startup … | 34 | 1832 | active |
| GestaltCogTeam/BasicTS BasicTS is a Python benchmark library and toolkit for fair and scalable time series analysis, built on PyTorch. It supports forecasting, cl… | 68 | 1814 | active |
| trevin-creator/autoresearch-mlx An Apple Silicon (MLX) port of Karpathy's autoresearch that runs autonomous AI research loops on a Mac without PyTorch or CUDA. A coding ag… | 55 | 1813 | active |
| AngusKit/AngusTester AngusTester is a self-hosted agile development and testing platform combining project management, functional/API/scenario testing, service … | 54 | 1807 | active |
| evalplus/evalplus EvalPlus is a Python framework and benchmark suite for rigorously evaluating LLM-synthesized code, providing HumanEval+ and MBPP+ datasets … | 41 | 1804 | active |
| 0vercl0k/wtf wtf (what the fuzz) is a distributed, code-coverage guided, snapshot-based fuzzer for attacking user- and kernel-mode targets on Windows an… | 74 | 1793 | active |
| evanphx/benchmark-ips A Ruby gem that enhances the standard Benchmark library with iterations-per-second (IPS) benchmarking. It automatically determines appropri… | 62 | 1773 | stable |
| SamSaffron/memory_profiler memory_profiler is a Ruby gem that reports memory usage and object allocations (allocated vs retained) by gem, file, location, and class. I… | 65 | 1766 | active |
| memtest86plus/memtest86plus Memtest86+ is a free, open-source, stand-alone memory tester for x86, x86-64, ARM AArch64, and LoongArch64 computers. It boots directly fro… | 82 | 1757 | active |
| Vchitect/VBench VBench is a comprehensive benchmark suite for evaluating video generative models across hierarchical evaluation dimensions, with prompt sui… | 61 | 1747 | active |
| decisionintelligence/TFB TFB is a comprehensive and fair benchmarking framework for time series forecasting methods, covering deep learning, machine learning, and s… | 68 | 1734 | active |
| cleverhans-lab/cleverhans CleverHans is a Python library for benchmarking machine learning systems' vulnerability to adversarial examples, providing reference implem… | 23 | 6450 | maintenance |
| martinus/nanobench ankerl::nanobench is a single-header, platform-independent microbenchmarking library for C++11/14/17/20. It measures ns/op, op/s, instructi… | 99 | 1730 | stable |
| sumerc/yappi Yappi is a fast tracing profiler for Python written in C that is aware of multithreading, asyncio, and gevent/greenlet concurrency. It supp… | 91 | 1725 | stable |
| coding-kitties/investing-algorithm-framework A Python framework for developing, backtesting, and deploying quantitative trading algorithms and trading bots. It supports vector and even… | 90 | 1714 | active |
| zai-org/ImageReward ImageReward is the first general-purpose human preference reward model for text-to-image generation, trained on 137k expert comparison pair… | 52 | 1702 | stable |
| HypoPG/hypopg HypoPG is a PostgreSQL extension written in C that adds support for hypothetical (virtual) indexes. It lets you test whether an index would… | 80 | 1696 | stable |
| pawurb/hotpath-rs hotpath-rs is an easy-to-configure Rust performance profiler that pinpoints where code spends time, burns CPU, and allocates memory. It ins… | 84 | 1695 | active |
| perftools/xhgui XHGui is a self-hosted web application that provides a graphical interface for viewing and analyzing XHProf profiling data collected from P… | 77 | 1687 | active |
| JonMagon/KDiskMark KDiskMark is an open-source HDD and SSD benchmark tool for Linux with a friendly Qt-based GUI, acting as a frontend to the Flexible I/O Tes… | 84 | 1682 | active |
| robocasa/robocasa RoboCasa is a large-scale simulation framework for training and benchmarking generally capable robots on everyday tasks, focused on kitchen… | 81 | 1677 | active |
| google/vizier Open Source Vizier is a Python-based service and library for black-box and hyperparameter optimization, based on Google's internal Vizier t… | 79 | 1671 | active |
| mattpocock/evalite Evalite is a TypeScript-native eval runner for testing LLM-powered applications, built on Vitest with a local dev server UI for exploring o… | 60 | 1669 | active |
| mourner/flamebearer A command-line tool that analyzes JavaScript CPU profiles from Chrome DevTools Performance recordings and Node .cpuprofile files, printing … | 92 | 1661 | active |
| tikv/pprof-rs pprof-rs is a CPU profiler library for Rust programs, built on top of backtrace-rs. It samples running threads and produces stack-trace rep… | 67 | 1649 | active |
| tevador/RandomX RandomX is a proof-of-work algorithm optimized for general-purpose CPUs that uses random code execution and memory-hard techniques to resis… | 92 | 1643 | stable |
| NVIDIA/nccl-tests NVIDIA's official test suite for NCCL that measures the performance and correctness of collective communication operations across GPUs. It … | 76 | 1636 | active |
| facebookresearch/DomainBed DomainBed is a PyTorch suite containing benchmark datasets and algorithms for domain generalization research. It implements many algorithms… | 10 | 1630 | active |
| mlcommons/inference Reference implementations of the MLPerf Inference benchmark suite, the industry-standard benchmarks for measuring how fast ML systems run m… | 83 | 1621 | active |
| clinicjs/node-clinic Clinic.js is an open-source Node.js performance profiling suite that diagnoses issues like event-loop delays, I/O bottlenecks, and memory l… | 23 | 5953 | maintenance |
| NVIDIA/RULER RULER is a benchmark framework from NVIDIA that generates synthetic test examples to measure the true effective context size of long-contex… | 66 | 1609 | active |
| bamlab/flashlight Flashlight is a CLI tool that audits Android apps (native, React Native, Flutter) and generates a performance score from aggregated metrics… | 67 | 1602 | active |
| nschloe/tuna Tuna is a lightweight Python profile viewer that visualizes runtime (cProfile) and import profiles in the browser using d3. It is a modern,… | 89 | 1595 | active |
| google/uncertainty-baselines A library of high-quality, minimal-dependency implementations of standard and state-of-the-art uncertainty and robustness methods for deep … | 77 | 1592 | active |
| jonasmr/microprofile Microprofile is an embeddable CPU and GPU profiler written in a few C++ files that you drop into your project. It provides scoped timers, c… | 67 | 1590 | active |
| web-arena-x/webarena WebArena is a standalone, self-hostable web environment and benchmark for building and evaluating autonomous web agents. It provides realis… | 43 | 1588 | active |
| facebook/CacheLib CacheLib is a C++ library providing a pluggable, thread-safe in-process caching engine with transparent DRAM and SSD tiering. It includes C… | 99 | 1585 | active |
| photosynthesis-team/piq PyTorch Image Quality (PIQ) is a collection of measures and metrics for image quality assessment in image-to-image tasks, written in pure P… | 23 | 1574 | stable |
| google-deepmind/bsuite bsuite (Behaviour Suite for Reinforcement Learning) is a collection of carefully-designed experiments from DeepMind that investigate core c… | 82 | 1555 | stable |
| vllm-project/guidellm GuideLLM is an SLO-aware benchmarking and evaluation platform for LLM inference deployments, simulating real-world workloads against OpenAI… | 90 | 1547 | active |
| alibaba/FederatedScope FederatedScope is a comprehensive federated learning platform built on PyTorch with an event-driven architecture. It provides convenient us… | 23 | 1540 | active |
| RightNow-AI/autokernel AutoKernel is an open-source autoresearch pipeline that takes any PyTorch model, profiles it to find GPU kernel bottlenecks, extracts them … | 48 | 1535 | active |
| gradle/gradle-profiler A command-line tool from Gradle that automates gathering profiling and benchmarking data for Gradle builds. It measures build execution tim… | 88 | 1530 | active |
| skforecast/skforecast Skforecast is a Python library for time series forecasting that turns any scikit-learn compatible estimator (LightGBM, XGBoost, CatBoost, K… | 95 | 1527 | active |
| bheisler/criterion.rs Criterion.rs is a statistics-driven microbenchmarking library for Rust that measures and analyzes code performance with rigorous statistica… | 67 | 5520 | maintenance |
| bencheeorg/benchee Benchee is an easy-to-use and extensible microbenchmarking library for Elixir. It runs functions with warmup, measures run time and memory … | 78 | 1514 | stable |
| uber/ADR ADR (Agentic AI Detection and Response) is an open-source enterprise security framework for AI agents, providing endpoint discovery of AI t… | 68 | 1501 | active |
| bestiejs/benchmark.js Benchmark.js is a JavaScript benchmarking library that supports high-resolution timers and returns statistically significant results. It po… | 10 | 5456 | maintenance |
| adsr/phpspy phpspy is a low-overhead sampling profiler for PHP 7+ written in C. It can attach to running PHP processes (CLI, Apache, FPM) without code … | 60 | 1500 | active |
| scientistproject/Scientist.net Scientist.net is a .NET library for carefully refactoring critical code paths by running old and new implementations side by side in produc… | 60 | 1488 | active |
| cheind/py-motmetrics py-motmetrics is a Python library for evaluating multiple object tracking (MOT) results with MOTChallenge-aligned CLEAR MOT, Identity, and … | 65 | 1487 | active |
| OpenGenerativeAI/llm-colosseum LLM Colosseum is a benchmarking tool that pits large language models against each other in real-time Street Fighter III matches to evaluate… | 28 | 1484 | active |
| callstack/reassure Reassure is a performance testing library for React and React Native that automates performance regression testing. It measures render dura… | 91 | 1479 | active |
| reczoo/FuxiCTR FuxiCTR is an open-source Python library for click-through rate (CTR) prediction built on PyTorch and TensorFlow. It offers a configurable,… | 77 | 1470 | active |
| timescale/tsbs TSBS is a Go-based benchmark suite for generating time series datasets and measuring read/write performance across many time series databas… | 70 | 1450 | active |
| brianfrankcooper/YCSB The Yahoo! Cloud Serving Benchmark (YCSB) is a Java framework and CLI tool for benchmarking NoSQL and cloud data stores against standardize… | 66 | 5229 | maintenance |
| ionelmc/pytest-benchmark A pytest plugin that provides a benchmark fixture for measuring code performance within tests. It automatically calibrates test rounds to t… | 88 | 1447 | stable |
| nucliweb/webperf-snippets A curated collection of JavaScript snippets for measuring and debugging web performance (Core Web Vitals, loading, and interaction metrics)… | 89 | 1444 | active |
| nvzqz/divan Divan is a fast and simple benchmarking library for Rust projects, used via a #[divan::bench] attribute and cargo bench. It offers allocati… | 70 | 1442 | active |
| boost-ext/ut UT (μt) is a C++20 micro unit testing framework distributed as a single header or C++20 module. It provides a macro-free, DSL-style API for… | 60 | 1437 | active |
| ferd/recon Recon is an Erlang library providing safe functions and scripts for diagnosing and debugging Erlang systems in production. It allows inspec… | 67 | 1427 | stable |
| evo-hq/evo Evo is a plugin for agentic coding frameworks (Claude Code, Codex, Cursor, etc.) that turns a codebase into an autoresearch loop: it discov… | 76 | 1421 | active |
| Lyken17/pytorch-OpCounter THOP (PyTorch-OpCounter) is a Python library that counts the MACs/FLOPs and parameters of PyTorch models. It profiles arbitrary nn.Modules … | 32 | 5078 | maintenance |
| NVIDIA/gdrcopy GDRCopy is a low-latency GPU memory copy library built on NVIDIA GPUDirect RDMA technology, allowing CPU-driven copies between host and GPU… | 84 | 1412 | active |
| microsoft/diskspd DiskSpd is a command-line storage load generator and performance testing tool from Microsoft's Windows and Cloud Server Infrastructure engi… | 60 | 1402 | active |
| nschloe/perfplot A Python library that extends timeit by benchmarking code snippets across varying input parameters and plotting the timing results. It is u… | 41 | 1387 | active |
| LouisBrunner/valgrind-macos A patched mirror of Valgrind that adds support for recent macOS versions, distributed via a Homebrew tap. It lets developers run memory err… | 77 | 1383 | active |
| Zefan-Cai/KVCache-Factory KVCache-Factory is a unified Python framework implementing many KV cache compression, retrieval, merging, and quantization methods for long… | 68 | 1375 | active |
| kengz/SLM-Lab SLM Lab is a modular deep reinforcement learning framework built in PyTorch, offering ready-to-use algorithms like PPO, SAC, DQN, and A2C v… | 95 | 1362 | active |
| nviennot/core-to-core-latency A Rust CLI tool that measures the latency of inter-core communication on a CPU by pinning threads to different cores and timing compare-exc… | 56 | 1360 | active |
| uxlfoundation/scikit-learn-intelex Intel's Extension for Scikit-learn is a free AI accelerator that speeds up existing scikit-learn workflows on CPUs and GPUs, claiming up to… | 91 | 1356 | active |
| appneta/tcpreplay Tcpreplay is a suite of GPLv3 command-line utilities for editing and replaying captured network traffic (pcap files) back onto the network … | 98 | 1344 | active |
| fenrus75/powertop PowerTOP is a Linux command-line tool for diagnosing power consumption and power management issues, with an interactive top-like TUI for ex… | 87 | 1331 | active |