microsoft/Olive

Health v2 · maintenance only

Adoption not part of the score

2382 stars · 312 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Olive is Microsoft's AI model optimization toolkit for the ONNX Runtime, automating finetuning, conversion, quantization, and compression of ML models for CPUs, GPUs, and NPUs. It composes optimization techniques into workflows that produce efficient ONNX models meeting accuracy and latency constraints for cloud or edge inference.

Use cases

• quantize an LLM for ONNX Runtime inference
• convert a Hugging Face model to optimized ONNX format
• finetune a small language model then optimize it for deployment
• optimize models to run on NPU hardware
• compress a model while preserving accuracy within a latency budget
• prepare models for edge inference on Windows devices

When to choose

• you deploy models with ONNX Runtime on CPU, GPU, or NPU
• you need automated quantization or conversion pipelines for ML models
• you want to finetune and optimize LLMs for Windows or edge devices

When to avoid

• you deploy with PyTorch or TensorFlow runtimes rather than ONNX
• you need a general-purpose training framework rather than an optimization toolkit
• your target hardware has no ONNX Runtime support

Facets

library · maturity active

machine-learning llm-training llm-inference cli gpu-computing machine-learning deep-learning large-language-models developer-tools gpu-computing python cli cross-platform windows onnx-runtime model-optimization quantization model-conversion npu finetuning microsoft linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="microsoft/Olive")