sqliteai/warp

Health v2 · maintenance only

Flags: young

Adoption not part of the score

2284 stars · 168 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

WARP is a dependency-free, embeddable C inference engine that runs trillion-parameter mixture-of-experts models like Kimi K3 by streaming activated expert weights directly from NVMe storage instead of holding them in RAM. It keeps shared weights in memory, uses a bounded expert cache, and achieves full-model local inference on consumer hardware such as a 64 GB MacBook Pro.

Use cases

• run the full Kimi K3 model locally on a 64 GB machine
• stream LLM weights from NVMe when the model exceeds available RAM
• embed a frontier-model inference engine in a C application with no dependencies
• experiment with mixture-of-experts weight paging and expert caching
• run large MoE models on consumer hardware without distillation or pruning

When to choose

• you want to run a huge MoE model locally and it does not fit in RAM
• you need a dependency-free, embeddable C inference engine
• you have fast NVMe storage and enough RAM for the shared trunk plus a cache
• you want to experiment with weight-streaming inference techniques

When to avoid

• you need high token throughput for interactive chat (K3 decodes at ~0.6 tok/s)
• you want broad model support or a mature ecosystem like llama.cpp
• your machine has less than 64 GB RAM for the largest models
• you need GPU-accelerated inference or multi-GPU serving

Facets

library · maturity active

llm-inference machine-learning large-language-models machine-learning developer-tools cross-platform cli mixture-of-experts nvme-streaming quantization embeddable c local-inference kimi-k3 macos linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="sqliteai/warp")