mlcommons/inference

Health v2 · maintenance only

Adoption not part of the score

1621 stars · 647 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Reference implementations of the MLPerf Inference benchmark suite, the industry-standard benchmarks for measuring how fast ML systems run models in datacenter and edge deployment scenarios. Maintained by MLCommons, it covers vision, language, recommendation, and generative AI workloads including LLMs and Stable Diffusion.

Use cases

• benchmark inference speed of GPUs and ML accelerators
• compare ML hardware performance across vendors
• run MLPerf inference benchmarks for a submission
• measure latency and throughput of LLM inference
• evaluate inference performance of vision models like ResNet and YOLO
• benchmark recommender system inference with DLRM

When to choose

• you need industry-standard, reproducible ML inference performance measurements
• you are preparing an official MLPerf submission
• you want to compare hardware or inference stacks fairly
• you need reference implementations for standard ML workloads

When to avoid

• you just want a simple microbenchmark for one custom model
• you need training performance benchmarks (use MLPerf Training instead)
• you want a lightweight profiling tool rather than a formal benchmark suite

Facets

library · maturity active

benchmarking machine-learning llm-inference machine-learning performance gpu-computing artificial-intelligence python cross-platform mlperf inference-benchmark hardware-evaluation reference-implementations performance-measurement linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="mlcommons/inference")