# evilsocket/cake

Distributed inference for mobile, desktop and server.

Repository: https://github.com/evilsocket/cake
Canonical: https://ross.abutalabs.com/products/evilsocket-cake
Language: Rust
License: NOASSERTION
License Family: other
Last push: 2026-04-24T15:11:28+00:00

## Health v2 (maintenance only)
Score: 59/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 79, release rhythm 35, longevity 56
- inputs: {"age_days": 787, "days_push": 131, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases, no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 3114, forks 207 (observed 2026-08-28T04:07:44.077400+00:00)

## What it is
Cake is a multimodal AI inference server written in Rust that runs text, image, and voice models on a single device or shards them across a heterogeneous cluster of phones, desktops, and servers. It exposes an OpenAI-compatible REST API with a built-in web UI and TUI chat client.

## Use cases
- run LLM inference on my own hardware
- shard a large model across multiple devices that don't have enough GPU memory
- self-host an OpenAI-compatible inference server
- generate images with Stable Diffusion or FLUX locally
- run text-to-speech with voice cloning
- cluster iPhones, Androids and desktops for AI inference

## When to choose
- you want to run models that exceed a single device's memory by pooling heterogeneous hardware
- you need multimodal inference (text, image, TTS) from one server
- you want zero-config mDNS-based clustering across platforms
- you prefer a Rust server with CUDA, Metal, Vulkan, and CPU backends

## When to avoid
- you need battle-tested, production-grade inference with long-term stability
- you require a permissive open-source license (it uses FAIR v1.0.0)
- you only need single-GPU inference where established servers suffice
- you need a wide ecosystem of plugins and integrations

## Facets
- artifact type: application
- maturity: experimental
- function: llm-inference, http-server, stable-diffusion, tts, chat-interface, gpu-computing
- domain: large-language-models, artificial-intelligence, image-processing, speech-processing, self-hosted
- platform: windows, rust, cli
- tags: distributed-inference, model-sharding, mdns-clustering, openai-compatible-api, multimodal, huggingface, linux, macos, android, ios, docker, web-server, gpu

## Member repositories
- evilsocket/cake (main) score 59

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:07:44.077400+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T07:26:40.694062+00:00, confidence not recorded.
  - readme: https://github.com/evilsocket/cake (fetched 2026-08-28T04:07:44.077400+00:00, sha 2ee08564062e)
- Data as of 2026-08-30T08:39:29.467469+00:00.