# kvcache-ai/Mooncake

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

Repository: https://github.com/kvcache-ai/Mooncake
Canonical: https://ross.abutalabs.com/products/mooncake
Homepage: https://kvcache-ai.github.io/Mooncake/
Language: C++
License: Apache-2.0
License Family: permissive
Topics: inference, kvcache, llm, rdma, sglang, vllm, disaggregation, tokenspeed, reinforcement-learning, trt-llm
Last push: 2026-08-26T12:04:34+00:00

## Health v2 (maintenance only)
Score: 91/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 99, release rhythm 99, longevity 57
- inputs: {"age_days": 799, "days_push": 7, "days_rel": 7, "gap_med": 12, "n_releases_24m": 28}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 6398, forks 1132 (observed 2026-08-28T04:09:43.056954+00:00)

## What it is
Mooncake is a KVCache-centric disaggregated serving platform for LLM inference, originally built to serve Kimi by Moonshot AI. It separates prefill and decode clusters, pools KVCache across CPU, DRAM, and SSD resources via RDMA, and includes a Transfer Engine and distributed Mooncake Store.

## Use cases
- serve LLMs with high throughput under SLO constraints
- build a distributed KVCache pool for vLLM or SGLang
- disaggregate prefill and decode for long-context workloads
- transfer KVCache between GPU nodes over RDMA
- speed up rollout data transfer in disaggregated reinforcement learning
- share KV cache across inference instances to avoid recomputation

## When to choose
- you operate multi-node LLM inference at scale and need KVCache reuse
- you want prefill/decode disaggregation with vLLM, SGLang, or TRT-LLM
- you have RDMA-capable GPU clusters and long-context workloads
- you need a distributed cache backend for RL rollout or online training

## When to avoid
- you serve small models on a single GPU without RDMA hardware
- you need a simple turnkey inference server rather than serving infrastructure
- your deployment lacks high-speed networking between nodes

## Facets
- artifact type: service
- maturity: active
- function: llm-inference, caching, gpu-computing, streaming, scheduling
- domain: large-language-models, artificial-intelligence, microservices, gpu-computing, infrastructure-as-code
- platform: cpp, python, self-hosted
- tags: kvcache, rdma, disaggregation, prefill-decode-separation, vllm, sglang, kv-cache-pool, llm-serving, linux, docker, gpu

## Member repositories
- kvcache-ai/Mooncake (main) score 91

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:09:43.056954+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T17:45:01.762004+00:00, confidence not recorded.
  - readme: https://github.com/kvcache-ai/Mooncake (fetched 2026-08-28T04:09:43.056954+00:00, sha 371d31382c66)
  - homepage: https://kvcache-ai.github.io/Mooncake/ (fetched 2026-08-29T08:41:52.140606+00:00, sha 3159b482bb6f)
- Data as of 2026-08-30T08:39:29.467469+00:00.