iree-org/iree

Health v2 · maintenance only

Adoption not part of the score

3901 stars · 989 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

IREE is an MLIR-based end-to-end machine learning compiler and runtime that lowers models from frameworks like PyTorch, TensorFlow, JAX, and ONNX into a unified intermediate representation. It compiles ahead-of-time to efficient binaries for CPUs, GPUs, and accelerators, scaling from datacenter deployments down to mobile, edge, and bare-metal targets.

Use cases

• compile pytorch models to run on gpu
• deploy machine learning models on mobile and edge devices
• run onnx models with low overhead runtime
• compile ml models for vulkan or cuda accelerators
• run ml inference on bare-metal embedded systems
• benchmark ml model inference across hardware targets
• convert tensorflow lite models to native binaries

When to choose

• you need ahead-of-time compilation of ML models with small binary sizes (down to ~30KB) for embedded or edge targets
• you need to deploy models across diverse hardware including CUDA, ROCm/HIP, Vulkan, and Metal from a single toolchain
• you need dynamic shapes, streaming, and control flow support in compiled models
• you want profiling, debugging, and bare-metal deployment support for ML workloads

When to avoid

• you just need a simple inference runtime for one framework on one platform - framework-native runtimes are simpler
• you need training support - IREE focuses on inference deployment, not training
• you want a turnkey solution without understanding compilers or MLIR concepts
• your target platform is not in the support matrix and you cannot contribute support

Facets

library · maturity active

compiler machine-learning llm-inference gpu-computing interpreter benchmarking machine-learning compilers deep-learning gpu-computing embedded-systems developer-tools windows cross-platform wasm cpp python mlir ml-model-deployment aot-compilation vulkan cuda rocm metal spirv edge-deployment hardware-abstraction-layer linux macos android ios gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="iree-org/iree")