luminal-ai/luminal

Health v2 · maintenance only

Flags: prerelease_only

Adoption not part of the score

2956 stars · 225 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Luminal is a high-performance general-purpose ML inference compiler written in Rust that lowers models to a minimal 15-op dataflow IR and compiles them to optimized GPU/ASIC kernels. It integrates natively with PyTorch as a torch.compile backend and also offers a Rust tensor API, running models like Llama 3 8B near theoretical peak performance on H100s.

Use cases

• run llama 3 8b locally on cuda with maximum speed
• compile pytorch models to optimized gpu kernels
• serve llm inference at near-theoretical-peak throughput on h100
• avoid heavyweight ml frameworks with a minimal rust tensor library
• automatically discover kernel fusions like flash attention without hand-written ops
• compile transformers and convnets ahead of time for gpu or asic

When to choose

• you need maximum inference throughput on NVIDIA GPUs and are comfortable with Rust or PyTorch integration
• you want a small, hackable ML compiler core instead of a large framework
• you want automatic kernel optimization (fusion, tiling, scheduling) via search rather than heuristics

When to avoid

• you need broad production support for many accelerators, quantization formats, or serving features today
• you rely on a mature ecosystem of ops, model zoos, and tooling like PyTorch or vLLM
• you need CPU-only or non-CUDA hardware support as a primary target

Facets

library · maturity active

llm-inference compiler machine-learning deep-learning gpu-computing machine-learning deep-learning large-language-models compilers gpu-computing performance rust python windows inference-compiler cuda pytorch-backend kernel-fusion tensor-api aot-compilation gpu linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="luminal-ai/luminal")