jundot/omlx

Health v2 · maintenance only

Adoption not part of the score

20760 stars · 1746 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

oMLX is a native macOS menu bar application and LLM inference server built on Apple's MLX framework, featuring continuous batching and two-tier (RAM + SSD) KV cache persistence. It exposes OpenAI- and Anthropic-compatible APIs so local models work as drop-in backends for tools like Claude Code and Cursor.

Use cases

• run local llm server on apple silicon mac
• serve mlx models with openai-compatible api
• speed up claude code with local llm backend
• persist kv cache across llm requests
• manage llm inference server from macos menu bar
• host multiple llm models with automatic memory eviction
• self-host anthropic api compatible endpoint

When to choose

• you have an Apple Silicon Mac and want fast local LLM serving with minimal setup
• you use coding agents like Claude Code or Cursor and want low time-to-first-token on repeated prefixes
• you want a native menu bar app with dashboard, auto-updates, and multi-model serving
• you need OpenAI and Anthropic drop-in API compatibility with tool calling and MCP support

When to avoid

• you need inference on Linux, Windows, or non-Apple-Silicon hardware
• you require CUDA or multi-GPU cluster serving at scale
• you need model architectures not supported by MLX
• you want a headless server without macOS

Facets

application · maturity active

llm-inference http-server caching api-framework chat-interface mcp large-language-models artificial-intelligence developer-tools self-hosted machine-learning python cli mlx apple-silicon kv-cache continuous-batching menu-bar-app openai-compatible anthropic-api local-llm ssd-caching homebrew macos web-server

Member repositories

For agents

markdown · JSON · MCP: product_card(name="jundot/omlx")