FareedKhan-dev/kimi-k3-in-c

Health v2 · maintenance only

Flags: young

Adoption not part of the score

6524 stars · 1062 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

A dependency-free C99 inference engine that runs the 2.78-trillion-parameter Kimi K3 model on a single CPU with as little as 8 GB of RAM by streaming quantized (MXFP4) weights from disk. The entire engine is ~176 KB with no BLAS, framework, or GPU requirement, using SIMD (AVX2), mixture-of-experts, and linear attention.

Use cases

• run a trillion-parameter LLM on a laptop without a GPU
• llm inference in pure c with no dependencies
• run kimi k3 locally on cpu
• study how transformer inference engines are built from scratch
• quantized mxfp4 model inference with low memory
• cpu-only inference with simd avx2 optimization

When to choose

• you want to run a very large LLM on commodity CPU hardware with no GPU
• you need a minimal, zero-dependency, portable C inference engine to read or modify
• you want to learn how MoE, linear attention, and quantized inference work at the systems level

When to avoid

• you need fast, production-grade token throughput - even high-RAM machines take seconds per token
• you want GPU acceleration or a mature ecosystem of tooling and integrations
• you need multi-user serving or an API server rather than single-prompt local inference

Facets

library · maturity active

llm-inference machine-learning deep-learning large-language-models deep-learning machine-learning developer-tools c cli cross-platform cpu-inference mixture-of-experts linear-attention quantization mxfp4 simd avx2 zero-dependencies transformer memory-efficient from-scratch systems-programming linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="FareedKhan-dev/kimi-k3-in-c")