JustVugg/colibri

Health v2 · maintenance only

Flags: young

Adoption not part of the score

26251 stars · 2872 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Colibrì is a pure-C, zero-dependency inference engine that runs frontier Mixture-of-Experts models (744B to 2.8T parameters) on consumer hardware by streaming expert weights from disk and treating VRAM, RAM, and storage as a single memory hierarchy. It ships per-model engines with a unified front end (chat, serve, web dashboard) and validates outputs token-exact against reference implementations.

Use cases

• run trillion-parameter MoE models on a consumer PC
• stream LLM expert weights from NVMe instead of loading into RAM
• self-host large language models without a datacenter
• serve a 744B model from a C engine with no dependencies
• run frontier models on Apple Silicon via Metal
• inspect which MoE experts fire with a live dashboard
• run LLM inference on CPU with limited VRAM

When to choose

• you want to run very large MoE models on hardware with limited VRAM/RAM
• you need a dependency-free, pure-C inference engine you can audit and hack on
• you want token-exact outputs matching reference implementations
• you're researching inference-side systems: memory tiering, prefetching, CPU/GPU overlap

When to avoid

• you need guaranteed latency or an SLA for production serving
• you need broad model support beyond the six supported families
• you want a mature ecosystem with extensive tooling and integrations
• you need fast throughput on a single consumer machine — streaming from disk trades speed for capacity

Facets

library · maturity active

llm-inference llm-training gpu-computing cli http-server chat-interface large-language-models machine-learning artificial-intelligence developer-tools self-hosted windows cpp cli self-hosted cross-platform mixture-of-experts inference-engine expert-streaming memory-tiering pure-c zero-dependencies quantization int4 nvme-streaming cpu-inference apple-silicon metal linux macos gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="JustVugg/colibri")