GradientHQ/parallax

Health v2 · maintenance only

Adoption not part of the score

1368 stars · 146 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Parallax is a distributed model serving framework that lets you build your own AI cluster across heterogeneous, geographically dispersed devices. It shards LLMs via pipeline parallelism and uses P2P communication to serve models like Llama, Qwen, and DeepSeek on personal hardware.

Use cases

• host a local LLM across my own devices
• build a private AI cluster from spare machines
• serve large models that don't fit on one GPU
• run decentralized LLM inference without a datacenter
• shard an LLM across Mac and Linux nodes
• self-host a chatbot backend on consumer hardware

When to choose

• you want to pool GPUs across multiple machines or locations to serve large LLMs
• you need decentralized, self-hosted inference without centralized cloud infrastructure
• your nodes have heterogeneous hardware and you need dynamic scheduling and routing

When to avoid

• you need a single-GPU, single-node inference server with minimal setup
• you require battle-tested, production-grade serving at massive scale
• you need low-latency inference where P2P network hops would hurt performance

Facets

framework · maturity experimental

llm-inference agent-framework chatbot rpc scheduling large-language-models microservices artificial-intelligence self-hosted machine-learning python cross-platform self-hosted decentralized-inference pipeline-parallelism model-sharding p2p-networking llm-serving ai-cluster kv-cache continuous-batching macos linux docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="GradientHQ/parallax")