waybarrios/vllm-mlx

Health v2 · maintenance only

Adoption not part of the score

1618 stars · 231 forks observed · 2026-10-12

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

vllm-mlx is a vLLM-style LLM inference server for Apple Silicon Macs built on native MLX, exposing both OpenAI and Anthropic compatible APIs from a single process. It supports continuous batching, paged and SSD-tiered KV caching, multimodal models (vision, audio, TTS, STT), MCP tool calling, and structured output.

Use cases

• run local llm server on mac with openai api
• serve anthropic-compatible api for claude code locally
• self-host llm inference on apple silicon
• run vision language models to analyze images locally
• local speech-to-text and text-to-speech server
• batch concurrent llm requests with continuous batching
• use mcp tool calling with local models
• generate embeddings and rerank documents locally

When to choose

• you have an Apple Silicon Mac and want high-throughput local inference without model conversion
• you need both OpenAI and Anthropic API compatibility from one server
• you want multimodal serving (text, image, audio, TTS/STT) in a single process
• you want to point Claude Code or OpenAI SDKs at a local endpoint

When to avoid

• you need GPU inference on Linux or NVIDIA CUDA hardware
• you are on Windows or Intel Macs
• you need a battle-tested production server at massive scale
• you only need simple single-user local chat without API compatibility

Facets

service · maturity active

llm-inference http-server api-framework speech-recognition tts mcp chatbot rag large-language-models artificial-intelligence machine-learning speech-processing self-hosted apis python cli self-hosted apple-silicon mlx openai-compatible anthropic-compatible continuous-batching local-llm inference-server multimodal claude-code kv-cache macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="waybarrios/vllm-mlx")