# zml/zml

Any model. Any hardware. Zero compromise. Built with @ziglang / @openxla / MLIR / @bazelbuild

Repository: https://github.com/zml/zml
Canonical: https://ross.abutalabs.com/products/zml
Homepage: https://zml.ai
Language: Zig
License: Apache-2.0
License Family: permissive
Topics: ai, bazel, hpc, inference, xla, zig
Last push: 2026-08-26T21:19:11+00:00

## Health v2 (maintenance only)
Score: 72/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 48, longevity 51
- inputs: {"age_days": 715, "days_push": 7, "days_rel": 133, "gap_med": null, "n_releases_24m": 1}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 4003, forks 178 (observed 2026-08-28T04:08:32.117720+00:00)

## What it is
ZML is a production LLM inference stack written in Zig, built on MLIR and OpenXLA, that compiles models to run at peak performance across NVIDIA, AMD, Intel, TPU, and AWS Trainium hardware from a single codebase. It is Python-free and built with Bazel, with examples for running models like Llama 3 and Qwen from Hugging Face, local directories, or S3.

## Use cases
- serve llm inference on any gpu hardware
- run llama 3 locally without python
- deploy models on tpu or trainium
- compile ai models for amd rocm
- build a fast llm server in zig
- avoid vendor lock-in for ai inference

## When to choose
- you need peak inference performance across heterogeneous accelerators
- you want a Python-free, low-overhead inference stack
- you want one codebase targeting CUDA, ROCm, OneAPI, TPU, and Trainium

## When to avoid
- you need a quick Python-based inference setup with broad model ecosystem support
- your team is not comfortable with Bazel and Zig toolchains
- you need CPU-only lightweight inference

## Facets
- artifact type: framework
- maturity: active
- function: llm-inference, machine-learning, compiler, gpu-computing
- domain: large-language-models, machine-learning, gpu-computing, developer-tools
- platform: cross-platform
- tags: zig, mlir, xla, bazel, inference-stack, hardware-agnostic, python-free, huggingface, linux, macos, gpu

## Member repositories
- zml/zml (main) score 72

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:08:32.117720+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T18:24:06.495964+00:00, confidence not recorded.
  - readme: https://github.com/zml/zml (fetched 2026-08-28T04:08:32.117720+00:00, sha cb6134f6bebc)
  - homepage: https://zml.ai (fetched 2026-08-29T09:17:05.684211+00:00, sha ba482e866724)
- Data as of 2026-08-30T08:39:29.467469+00:00.