exo-explore/exo

Health v2 · maintenance only

Adoption not part of the score

47081 stars · 3464 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

exo is a Python application that turns a collection of devices (Macs, GPUs, etc.) into a distributed AI cluster for running large language models locally. It automatically shards models across devices using topology-aware parallelism and exposes OpenAI/Claude/Ollama-compatible APIs.

Use cases

• run large language models that don't fit on a single device
• cluster multiple Mac Studios to run DeepSeek or Llama locally
• serve local LLMs with an OpenAI-compatible API
• run frontier AI models offline without cloud services
• speed up local inference by adding more devices
• chat with local models through a web dashboard

When to choose

• you have multiple devices and want to run models too large for any single one
• you want local, private LLM inference with existing OpenAI-compatible clients
• you use Apple Silicon and want MLX-based distributed inference

When to avoid

• you only need to run small models on one device
• you need managed cloud inference or fine-tuning/training
• you need Windows-only heterogeneous GPU clusters with first-class support

Facets

application · maturity active

llm-inference machine-learning http-server chat-interface large-language-models artificial-intelligence microservices self-hosted python cross-platform self-hosted distributed-inference tensor-parallelism rdma mlx openai-api-compatible device-cluster local-ai macos linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="exo-explore/exo")