resource: benchmarking
233 resources, primary matches first, then adoption-weighted; health v2 shown.
| Resource | Health v2 | Stars | Maturity |
|---|---|---|---|
| engcang/SLAM-application A curated collection of installation guides, configuration files, and comparison results for many open-source LiDAR(-inertial) SLAM and odo… | 34 | 1206 | active |
| SJTU-ViSYS/M2DGR M2DGR is a multi-modal, multi-scenario SLAM dataset collected with ground robots, providing synchronized vision, lidar, IMU, GNSS, event, a… | 71 | 1194 | stable |
| Accio-org/CommerceAgentBench CommerceAgentBench is a benchmark of 107 long-horizon agent tasks evaluated in high-fidelity, stateful, reproducible replicas of real onlin… | 57 | 1192 | active |
| pdebench/PDEBench PDEBench is a benchmark suite for scientific machine learning consisting of code and large ready-to-use datasets of time-dependent partial … | 56 | 1191 | active |
| Azure/AzurePublicDataset A repository of public Microsoft Azure workload traces released for the research community, including VM workloads, Azure Functions invocat… | 88 | 1184 | active |
| deepseek-ai/profile-data A public release of PyTorch Profiler traces from DeepSeek's V3/R1 training and inference infrastructure, showing computation-communication … | 25 | 1182 | active |
| astavonin/go-optimization-guide A long-form technical guide (published at goperf.dev, built with MkDocs Material) covering patterns and techniques for writing high-perform… | 52 | 1134 | active |
| huggingface/search-and-learn Search and Learn is a collection of Python scripts and YAML recipes from Hugging Face for scaling inference-time (test-time) compute of ope… | 59 | 1131 | active |
| geektutu/high-performance-go An open-source e-book (with accompanying code) on high-performance Go programming, covering profiling, data structures, concurrency, and co… | 32 | 3887 | maintenance |
| AlphaGPU/leetgpu-challenges A collection of GPU programming challenges for the LeetGPU.com platform, containing problem descriptions, reference implementations, test c… | 63 | 1114 | active |
| bird-bench/BIRD-CRITIC-1 BIRD-CRITIC 1.0 (a.k.a. SWE-SQL) is a benchmark dataset of real-world SQL user issues for evaluating whether LLMs can diagnose and fix data… | 53 | 1098 | active |
| ClickHouse/ClickBench ClickBench is an open-source benchmark for analytical (OLAP) databases, built on a real anonymized web analytics traffic dataset with 43 re… | 77 | 1095 | active |
| google/cluster-data A repository hosting Google Borg cluster workload traces, including machine-level job and task data from eight Borg cells (May 2019), an ol… | 75 | 1095 | stable |
| Tongkaio/CUDA_Kernel_Samples A curated collection of hand-written CUDA kernel samples covering common operators like elementwise add, reduction, softmax, sgemm, and tra… | 42 | 1095 | active |
| PaddleJitLab/CUDATutorial A self-learning tutorial repository for CUDA high-performance programming, structured as progressive lesson series from beginner environmen… | 60 | 1087 | active |
| DataDog/go-profiler-notes A collection of in-depth notes by felixge on Go's profiling methods, including the pprof tool and format, stack traces, goroutine, and bloc… | 77 | 3665 | maintenance |
| Jingkang50/OpenOOD OpenOOD is a benchmark framework that reproduces and fairly compares methods for generalized out-of-distribution detection, spanning anomal… | 48 | 1070 | active |
| carlini/yet-another-applied-llm-benchmark A personal benchmark of nearly 100 applied tests for evaluating how well large language models perform on practical tasks the author has ac… | 34 | 1064 | active |
| lmarena/arena-hard-auto Arena-Hard-Auto is an automatic benchmark for evaluating instruction-tuned LLMs using challenging real-world prompts and LLM-based judges (… | 39 | 1060 | active |
| imteekay/web-performance-research A curated compilation of articles, videos, courses, and tools for learning web performance, covering topics like Core Web Vitals, rendering… | 55 | 1050 | active |
| aws/aws-graviton-getting-started A technical guide (published as an mdBook) from AWS for developers using Amazon EC2 instances powered by Graviton Arm processors, covering … | 76 | 1047 | active |
| pytorch/benchmark TorchBench is a collection of open source benchmarks used to evaluate PyTorch performance across popular model workloads. It provides a sta… | 77 | 1041 | active |
| xiaowu0162/LongMemEval LongMemEval is a benchmark of 500 high-quality questions for evaluating the long-term memory abilities of chat assistants across five skill… | 58 | 1035 | active |
| fabienrenaud/java-json-benchmark A JMH-based benchmark suite measuring serialization and deserialization throughput of dozens of Java JSON libraries. It publishes comparati… | 76 | 1034 | active |
| Minqi824/ADBench ADBench is the official implementation of a comprehensive tabular anomaly detection benchmark published at NeurIPS 2022, comparing 30 algor… | 60 | 1021 | active |
| bird-bench/BIRD-Interact BIRD-INTERACT is an interactive Text-to-SQL benchmark that evaluates LLMs through dynamic multi-turn interactions with a simulated user, a … | 52 | 1011 | active |
| X-EraAI/ActPhysCause-Challenge ActPhysCause Challenge is a benchmark and dataset for action-conditioned physical and causal world modeling, hosted as Track 0 of the LoViF… | 54 | 1004 | active |
| openai/human-eval HumanEval is a hand-written evaluation dataset of 164 Python programming problems with unit tests, plus a harness for measuring code-genera… | 33 | 3354 | maintenance |
| eishay/jvm-serializers A benchmark suite comparing the performance of serialization libraries on the JVM, such as protobuf, thrift, avro, and Java built-in serial… | 32 | 3283 | maintenance |
| google/BIG-bench BIG-bench is a collaborative benchmark with over 200 tasks for probing and measuring the capabilities of large language models. It includes… | 10 | 3247 | maintenance |
| google-deepmind/alphatensor Code and data accompanying DeepMind's AlphaTensor publication on discovering faster matrix multiplication algorithms with reinforcement lea… | 32 | 2853 | maintenance |
| campoy/go-tooling-workshop A multi-hour workshop by Francesc Campoy covering the Go tooling ecosystem, including source code management, building artifacts, and dynam… | 32 | 2775 | maintenance |
| FranxYao/chain-of-thought-hub A benchmarking hub and collection of evaluation scripts measuring large language models' complex reasoning ability using chain-of-thought p… | 30 | 2775 | maintenance |
| ept/hermitage Hermitage is a suite of test cases that probes how different database systems actually implement their transaction isolation levels, simula… | 61 | 2743 | maintenance |
| aaron-xichen/pytorch-playground A PyTorch playground providing pretrained models and dataset loaders for popular benchmarks like MNIST, SVHN, CIFAR10/100, STL10, and Image… | 32 | 2716 | maintenance |
| fenbf/AwesomePerfCpp A curated awesome-style list of C/C++ performance optimization resources including talks, articles, books, libraries, tools, and blogs. It … | 32 | 2552 | maintenance |
| wenbihan/reproducible-image-denoising-state-of-the-art A curated collection of popular and reproducible single-image denoising algorithms, spanning classical filtering, sparse coding, and deep l… | 32 | 2516 | maintenance |
| thlorenz/v8-perf A collection of notes and resources explaining V8 internals and how they affect Node.js and JavaScript performance. It covers data types, t… | 32 | 2283 | maintenance |
| deephacks/awesome-jvm A curated list of JVM low-level, performance-related, and non-framework resources, inspired by awesome-python. It catalogs tools and projec… | 32 | 2175 | maintenance |
| numenta/NAB The Numenta Anomaly Benchmark (NAB) is a benchmark for evaluating anomaly detection algorithms on streaming, real-time time-series data. It… | 23 | 2103 | maintenance |
| flame/how-to-optimize-gemm A tutorial wiki repository by Prof. Robert van de Geijn teaching step-by-step how to optimize general matrix-matrix multiplication (GEMM) u… | 32 | 2035 | maintenance |
| XiongjieDai/GPU-Benchmarks-on-LLM-Inference A curated benchmark dataset comparing LLM inference speeds (tokens/s) across many NVIDIA GPUs and Apple Silicon chips using llama.cpp on LL… | 28 | 1935 | maintenance |
| smallnest/1m-go-tcp-server A collection of Go TCP server implementations benchmarked at supporting 1 million concurrent connections, comparing goroutine-per-connectio… | 32 | 1928 | maintenance |
| szilard/benchm-ml A minimal benchmark comparing scalability, speed, and accuracy of open-source machine learning implementations (R packages, scikit-learn, H… | 32 | 1895 | maintenance |
| syhw/wer_are_we A curated, community-maintained tracker of state-of-the-art word error rate (WER) results on speech recognition benchmarks like LibriSpeech… | 32 | 1864 | maintenance |
| akutz/go-generics-the-hard-way A hands-on tutorial repository for learning Go generics introduced in Go 1.18, with labs, benchmarks, and an FAQ. It explains generic type … | 32 | 1863 | maintenance |
| addyosmani/puppeteer-webperf A collection of JavaScript recipes for automating web performance measurement using Puppeteer and headless Chrome. It includes runnable exa… | 32 | 1858 | maintenance |
| ChineseGLUE/ChineseGLUE ChineseGLUE is a language understanding evaluation benchmark for Chinese, providing datasets, baseline pre-trained models, corpora, and a p… | 32 | 1782 | maintenance |
| aliyun-node/Node.js-Troubleshooting-Guide A Chinese-language troubleshooting handbook for Node.js applications covering online/offline failure diagnosis, load-testing issues, and pe… | 32 | 1745 | maintenance |
| zanfranceschi/rinha-de-backend-2023-q3 Rinha de Backend 2023 Q3 is a community-run backend performance competition where participants build a people CRUD API and compete under st… | 40 | 1708 | maintenance |
| tczhangzhi/pytorch-distributed A collection of PyTorch example scripts demonstrating different distributed/multi-GPU training approaches (DataParallel, torch.distributed,… | 32 | 1655 | maintenance |
| ilkarman/DeepLearningFrameworks A collection of Jupyter notebooks implementing the same neural network tasks (CNN on CIFAR-10, RNN, etc.) across many deep learning framewo… | 32 | 1651 | maintenance |
| hendrycks/test The MMLU (Massive Multitask Language Understanding) benchmark dataset and evaluation code from Hendrycks et al., ICLR 2021. It contains mul… | 32 | 1613 | maintenance |
| CLUEbenchmark/CLUENER2020 CLUENER2020 is a fine-grained Chinese named entity recognition dataset and benchmark with 10 entity categories (address, book, company, gam… | 32 | 1521 | maintenance |
| google/fuzzer-test-suite A suite of fuzzing benchmarks derived from real-life C/C++ libraries with interesting bugs and hard-to-reach code paths, designed to evalua… | 10 | 1471 | maintenance |
| openai/SWELancer-Benchmark SWE-Lancer is a benchmark dataset and evaluation harness measuring whether frontier LLMs can complete real-world freelance software enginee… | 10 | 1431 | maintenance |
| facontidavide/CPP_Optimizations_Diary A blog-style repository documenting practical C++ optimization tips, design patterns, and benchmarking techniques. It emphasizes measuring … | 32 | 1427 | maintenance |
| oxwhirl/smac SMAC is a benchmark environment for cooperative multi-agent reinforcement learning built on StarCraft II via Blizzard's SC2 ML API and Deep… | 23 | 1366 | maintenance |
| arschles/go-in-5-minutes A collection of code samples, outlines, and videos for the 'Go in 5 Minutes' screencast series teaching practical Go programming topics. Ea… | 32 | 1348 | maintenance |
| LiYangHart/Hyperparameter-Optimization-of-Machine-Learning-Algorithms A collection of Jupyter Notebook examples implementing hyperparameter optimization techniques (grid search, random search, Bayesian optimiz… | 32 | 1342 | maintenance |
| devonestes/fast-elixir A collection of benchmarked Elixir code idioms comparing semantically equivalent approaches to show which is faster, inspired by fast-ruby.… | 32 | 1289 | maintenance |
| hendrycks/robustness Datasets and evaluation code for benchmarking neural network robustness to common corruptions and perturbations, from the ICLR 2019 paper b… | 32 | 1172 | maintenance |
| Escheee/TBCF A benchmark repository collecting and sharing results, papers, and code for correlation filter-based visual object trackers. It provides pe… | 32 | 1142 | maintenance |
| kickstarter/kickstarter-autodesk-3d A standardized 3D test geometry and assessment protocol co-developed by Autodesk and Kickstarter for evaluating FDM 3D printer performance.… | 32 | 1129 | maintenance |
| feiskyer/linux-perf-examples Source code examples for the Geek Time course 'Linux Performance Optimization in Practice' (《Linux 性能优化实战》). It provides reproducible C/Doc… | 32 | 1123 | maintenance |
| aras-p/ToyPathTracer A small educational path tracer written in C++ and C# with CPU and GPU implementations (DX11, Metal, Unity Burst, WebAssembly). It was buil… | 39 | 1116 | maintenance |
| Lyken17/Efficient-PyTorch A collection of best practices and example code for efficiently training large datasets like ImageNet with PyTorch. It demonstrates techniq… | 32 | 1103 | maintenance |
| Troyanovsky/Local-LLM-Comparison-Colab-UI A curated comparison of open-source LLMs that run on consumer hardware, with one-click Google Colab WebUI notebooks for trying each model. … | 57 | 1101 | maintenance |
| THU-LYJ-Lab/T3Bench T3Bench is the first comprehensive benchmark for text-to-3D generation, providing 300 text prompts across three complexity levels plus auto… | 27 | 1098 | maintenance |
| atemerev/skynet A multi-language microbenchmark that spawns one million actors/coroutines/futures in a recursive fan-out tree and sums their results, compa… | 32 | 1062 | maintenance |
| soumith/convnet-benchmarks A benchmarking suite and published results comparing publicly available implementations of convolutional neural networks (convnets) across … | 32 | 2686 | abandoned |
| jcjohnson/cnn-benchmarks A collection of benchmark results and scripts measuring inference speed of popular CNN models (AlexNet, VGG, ResNet, Inception) across CPUs… | 32 | 2532 | abandoned |
| jostmey/NakedTensor A set of bare-bones TensorFlow example scripts that teach the fundamentals by fitting a straight line to data using gradient descent. It pr… | 32 | 2405 | abandoned |
| zer0n/deepframeworks A research write-up comparing popular deep learning toolkits (Caffe, CNTK, TensorFlow, Theano, Torch) across modeling capability, performan… | 32 | 2038 | abandoned |
| ogrisel/parallel_ml_tutorial A Jupyter notebook tutorial on parallel machine learning with scikit-learn and IPython, covering scalable feature extraction, parallel cros… | 32 | 1589 | abandoned |
| smallnest/C1000K-Servers A collection of eight high-performance WebSocket server implementations (Spray-can, Netty, Undertow, Jetty, Vert.x, Grizzly, Node.js, Go) u… | 10 | 1504 | abandoned |
| Intel-bigdata/HiBench HiBench is a big data benchmark suite with 29 workloads across micro, machine learning, SQL, graph, websearch, and streaming categories for… | 10 | 1484 | abandoned |
| openai/grade-school-math GSM8K is a dataset of 8.5K linguistically diverse grade school math word problems (7.5K train, 1K test) released by OpenAI to diagnose mult… | 10 | 1451 | abandoned |
| xFFFFF/Gekko-Strategies A collection of trading strategies for the Gekko trading bot, complete with backtest results stored in a CSV database. It also bundles inst… | 32 | 1440 | abandoned |
| samccone/The-cost-of-transpiling-es2015-in-2016 A 2016 benchmark study comparing the bundle size and runtime overhead of transpiling ES2015 JavaScript to ES5 with different tools such as … | 32 | 1325 | abandoned |
| stephanenicolas/Quality-Tools-for-Android A sample Android app plus test suites demonstrating how to integrate Java quality-analysis and testing tools (Robolectric, Espresso, Roboti… | 32 | 1255 | abandoned |
| araffin/rl-baselines-zoo A collection of 100+ pre-trained reinforcement learning agents built with Stable Baselines, including tuned hyperparameters for each enviro… | 10 | 1203 | abandoned |
| ashfurrow/xcode-hardware-performance A collection of benchmark results measuring Xcode build times (fresh and incremental) across various Mac models, including Apple M1 and Int… | 10 | 1121 | abandoned |
| liguodongiot/llm-action A curated Chinese-language knowledge base covering large language model engineering, including training, fine-tuning, inference, compressio… | 71 | 24958 | active |
| stefan-jansen/machine-learning-for-trading Companion code repository for the book 'Machine Learning for Trading, 3rd Edition' by Stefan Jansen, containing 446+ Jupyter notebooks acro… | 86 | 20669 | active |
| NVIDIA/DeepLearningExamples A collection of state-of-the-art deep learning model training and deployment scripts from NVIDIA, organized by model and optimized for Tens… | 32 | 14840 | active |
| zalandoresearch/fashion-mnist Fashion-MNIST is a dataset of 70,000 28x28 grayscale Zalando article images across 10 classes, designed as a direct drop-in replacement for… | 32 | 12810 | stable |
| antonputra/tutorials A collection of companion code and lessons for Anton Putra's DevOps video tutorials on YouTube, covering Kubernetes, Terraform, Ansible, Ar… | 69 | 4677 | active |
| goldbergyoni/nodejs-testing-best-practices A comprehensive guide to modern Node.js testing featuring 50+ best practices, advanced topics like database, message queue, and contract te… | 52 | 4390 | active |
| PolymathicAI/the_well The Well is a 15TB collection of machine learning datasets containing numerical simulations of spatiotemporal physical systems, spanning 16… | 62 | 4389 | active |
| zzsting88/relayAPI A curated Chinese-language guide and evaluation of AI API relay/proxy stations (中转站) that resell access to GPT, Claude, Gemini, DeepSeek an… | 59 | 4179 | active |
| alsotang/node-lessons A Chinese-language Node.js tutorial course ('Node.js 包教不包会') by alsotang, consisting of 19 hands-on lessons covering Express, web scraping,… | 32 | 16434 | maintenance |
| verazuo/jailbreak_llms A research dataset accompanying the ACM CCS 2024 paper 'Do Anything Now', containing 15,140 ChatGPT prompts collected from Reddit, Discord,… | 28 | 3791 | stable |
| rlcode/reinforcement-learning A collection of minimal, clean, one-file-per-algorithm reinforcement learning examples in Python, covering classic methods like Q-Learning … | 71 | 3659 | active |
| zjhellofss/KuiperInfer KuiperInfer is an open-source course that guides learners through building a high-performance deep learning inference engine from scratch i… | 46 | 3496 | active |
| jonyzhang2023/awesome-embodied-vla-va-vln A curated awesome-list of 700+ state-of-the-art research papers and resources in embodied AI, covering Vision-Language-Action (VLA) models,… | 64 | 3481 | active |
| BBuf/how-to-optim-algorithm-in-cuda A curated collection of notes and hands-on code for optimizing algorithms on CUDA GPUs, covering handwritten kernels, CUTLASS/CuTe, Triton,… | 98 | 3223 | active |
| pibigstar/go-demo A Go language example tutorial repository covering basics through advanced topics, including standard library usage, design patterns, inter… | 34 | 2709 | active |
| mrgloom/awesome-semantic-segmentation A curated awesome-list of resources for semantic segmentation in deep learning, including papers, network architectures (U-Net, etc.), impl… | 32 | 10847 | maintenance |
| OpenCoder-llm/OpenCoder-llm OpenCoder is a fully open and reproducible family of code large language models (1.5B and 8B base and chat variants) trained on 2.5 trillio… | 22 | 2111 | active |