microsoft/onnxruntime

Health v2 · maintenance only

Adoption not part of the score

21654 stars · 4176 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

ONNX Runtime is a cross-platform, high-performance machine-learning accelerator for running inference and training on ONNX models. It supports models from PyTorch, TensorFlow, and scikit-learn, and optimizes execution across CPU, GPU, and NPU hardware via an extensible execution-provider framework.

Use cases

• run onnx models with high performance inference
• deploy pytorch models in c++ or c# apps
• accelerate transformer training on nvidia gpus
• run machine learning models on mobile and edge devices
• run llm inference locally in the browser
• speed up scikit-learn and xgboost model inference
• on-device training of models

When to choose

• you need fast, production-grade inference across many platforms and languages
• you want to train in Python but deploy in C#, C++, Java, or JavaScript
• you need to leverage diverse hardware accelerators (CUDA, TensorRT, DirectML, CoreML, NNAPI, QNN) through one runtime
• you want to run generative AI or LLM models on-device for privacy and cost savings

When to avoid

• your models stay in the original framework and framework-native inference is fast enough
• you need training features beyond what ORT's training support covers (e.g., complex custom training loops)
• you depend on a hardware accelerator only available as a preview or deprecated execution provider

Facets

library · maturity stable

machine-learning llm-inference deep-learning gpu-computing llm-training machine-learning deep-learning large-language-models artificial-intelligence cross-platform windows browser python cpp cross-platform wasm onnx inference-engine execution-providers hardware-acceleration model-optimization edge-ai pytorch tensorflow scikit-learn directml cuda tensorrt linux macos android ios gpu nodejs

Member repositories

For agents

markdown · JSON · MCP: product_card(name="microsoft/onnxruntime")