Open-source benchmarking software

817 open-source libraries, tools and applications tagged benchmarking, ranked by how closely benchmarking is their primary purpose, then by adoption and maintenance health. Each entry links to a page with sources, use cases and when to choose or avoid it.

817 products, primary matches first, then adoption-weighted; health v2 shown.

Product	Health v2	Stars	Maturity
Gumpest/YOLOv5-Multibackbone-Compression A YOLOv5-based toolbox for swapping in lightweight or high-accuracy backbones (TPH-YOLOv5, GhostNet, ShuffleNetV2, MobileNetV3-Small, Effic…	32	1020	maintenance
allegro/allRank allRank is a PyTorch-based framework for training neural learning-to-rank (LTR) models. It provides pointwise, pairwise, and listwise loss …	32	1011	maintenance
yizhang7210/liang Liang is a Python library for specifying non-functional requirements like latency thresholds directly in function interfaces via decorators…	32	1010	maintenance
PhyAgentOS/PhyAgentOS-core PhyAgentOS is a self-evolving embodied AI operating system built on agentic workflows, providing a session-centered runtime that decouples …	68	2511	experimental
Future AGI Future AGI is an open-source, self-hostable platform for evaluating, observing, and improving LLM and AI agent applications, combining trac…	77	2146	experimental
YiVal/YiVal YiVal is an open-source Python framework that automatically tunes prompts, RAG configurations, and model parameters for generative AI appli…	20	2137	experimental
EthicalML/xai XAI is a Python library from the Institute for Ethical AI & ML that provides tools for analysing and evaluating machine learning datasets a…	59	1267	experimental
moonbitlang/core moonbitlang/core is the standard library for the MoonBit programming language, released alongside the compiler. It provides builtin types, …	71	1226	experimental
trotsky1997/MathBlackBox MathBlackBox (MCTSr) is a Python research implementation of Monte Carlo Tree Self-Refine, an algorithm that combines MCTS with LLM self-ref…	24	1032	experimental
OptiMate OptiMate is a collection of Python libraries from Nebuly AI for optimizing AI model performance, including Speedster for inference accelera…	23	8328	abandoned
Glimpse Glimpse is an open-source diagnostics platform for ASP.NET web applications that inspects requests in real time, profiling server-side acti…	10	2310	abandoned
IntelLabs/fastRAG fastRAG is a Python research framework for building efficient, optimized retrieval-augmented generation (RAG) pipelines, built on Haystack …	10	1786	abandoned
tomlepaine/fast-wavenet A Python/TensorFlow library implementing an efficient O(L) generation algorithm for Wavenet-style autoregressive models using dynamic progr…	32	1769	abandoned
openresty/openresty-systemtap-toolkit A collection of SystemTap-based real-time analysis and diagnostics tools for OpenResty, NGINX, LuaJIT, and ngx_lua. It provides scripts for…	32	1668	abandoned
ELS-RD/kernl Kernl is an open-source PyTorch model optimization library that accelerates transformer inference on GPUs with a single line of code, using…	10	1583	abandoned
Atropos Atropos is a Python framework from Nous Research for building reinforcement learning environments that collect and evaluate LLM trajectorie…	10	1351	abandoned
coreylynch/async-rl A TensorFlow + Keras implementation of asynchronous 1-step Q-learning from the DeepMind 'Asynchronous Methods for Deep Reinforcement Learni…	32	1003	abandoned