evilsocket/cake

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

3126 stars · 206 forks observed · 2026-10-13

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Cake is a multimodal AI inference server written in Rust that runs text, image, and voice models on a single device or shards them across a heterogeneous cluster of phones, desktops, and servers. It exposes an OpenAI-compatible REST API with a built-in web UI and TUI chat client.

Use cases

• run LLM inference on my own hardware
• shard a large model across multiple devices that don't have enough GPU memory
• self-host an OpenAI-compatible inference server
• generate images with Stable Diffusion or FLUX locally
• run text-to-speech with voice cloning
• cluster iPhones, Androids and desktops for AI inference

When to choose

• you want to run models that exceed a single device's memory by pooling heterogeneous hardware
• you need multimodal inference (text, image, TTS) from one server
• you want zero-config mDNS-based clustering across platforms
• you prefer a Rust server with CUDA, Metal, Vulkan, and CPU backends

When to avoid

• you need battle-tested, production-grade inference with long-term stability
• you require a permissive open-source license (it uses FAIR v1.0.0)
• you only need single-GPU inference where established servers suffice
• you need a wide ecosystem of plugins and integrations

Facets

application · maturity experimental

llm-inference http-server stable-diffusion tts chat-interface gpu-computing large-language-models artificial-intelligence image-processing speech-processing self-hosted windows rust cli distributed-inference model-sharding mdns-clustering openai-compatible-api multimodal huggingface linux macos android ios docker web-server gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="evilsocket/cake")