ONNX Runtime

Health v2 · maintenance only

Adoption not part of the score

22115 stars · 4332 forks observed · 2026-10-19

What it is AI-extracted, prompt v1, taxonomy v1, 2026-10-12, confidence 0.97

ONNX Runtime is a high‑performance, cross‑platform library for executing ONNX machine‑learning models. It supports both inference and training across many languages and hardware accelerators.

Use cases

• Accelerate inference of deep learning models on CPU or GPU
• Run ONNX models on edge devices such as mobile phones or IoT hardware
• Integrate pretrained PyTorch or TensorFlow models into C++ or Java applications
• Scale generative AI model serving across cloud and on‑premise environments
• Benchmark and compare performance of different hardware execution providers

When to choose

• You need a single runtime that works on many operating systems and hardware back‑ends
• Your models are exported to ONNX format and you want optimal performance without rewriting code
• You require support for multiple programming languages (Python, C++, C#, Java, JavaScript, etc.)
• You want to leverage hardware‑specific accelerators like CUDA, TensorRT, DirectML, or NPU

When to avoid

• Your workflow relies on a framework that does not export to ONNX
• You need fine‑grained control over custom kernels beyond what execution providers expose
• You prefer a pure Python inference library with minimal native dependencies

Facets

library · maturity active

machine-learning artificial-intelligence machine-learning deep-learning windows onnx inference training hardware-acceleration cross-platform gpu cpu linux macos android ios web-browser nodejs

Member repositories

For agents

markdown · JSON · MCP: product_card(name="microsoft/onnxruntime")