trymirai/uzu

Health v2 · maintenance only

Adoption not part of the score

1678 stars · 73 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Uzu is a high-performance inference engine written in Rust for running AI models directly on-device, with Python, TypeScript, and Swift bindings. It leverages Apple Silicon's unified memory and Metal to deliver zero-latency, private, cost-free inference inside applications.

Use cases

• run llm inference locally on apple silicon
• embed an ai model directly in an ios or macos app
• serve a local api for chatting with models
• generate structured output from a local model
• run text-to-speech on device
• avoid cloud inference costs and keep data private
• deploy ai in an app with zero latency

When to choose

• you target Apple Silicon (macOS/iOS) and want fast on-device inference
• you need full data privacy with no data leaving the device
• you want to eliminate per-token cloud inference costs
• you want simple high-level APIs across Rust, Python, TypeScript, or Swift
• you need low-latency, interactive AI experiences in your app

When to avoid

• you need to run on non-Apple hardware such as Linux, Windows, Android, or CUDA GPUs
• you need broad cross-platform model support beyond the curated model list
• you require serving many concurrent users with batched throughput rather than batch size 1
• you need a mature ecosystem like llama.cpp or vLLM with extensive community tooling

Facets

library · maturity active

llm-inference sdk tts machine-learning large-language-models machine-learning artificial-intelligence cross-platform rust python on-device-ai metal apple-silicon local-inference quantization unified-memory edge-ai privacy-preserving structured-output chat-completions macos ios swift nodejs gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="trymirai/uzu")