ikawrakow/ik_llama.cpp

Health v2 · maintenance only

Flags: prerelease_only

Adoption not part of the score

3304 stars · 490 forks observed · 2026-10-17

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A fork of llama.cpp offering state-of-the-art quantization types and improved CPU/CUDA inference performance for large language models. It pioneered features like MLA, quant repacking, and tensor parallelism before they landed upstream.

Use cases

• run LLMs locally with faster CPU inference
• quantize models with better quality-per-bit formats
• run MoE models with experts on CPU and attention on GPU
• get better prompt processing speed than mainline llama.cpp
• run DeepSeek-style MLA models efficiently
• serve LLMs on consumer hardware with limited VRAM

When to choose

• you want maximum CPU or hybrid CPU/GPU inference performance for LLMs
• you need advanced quantization types not in mainline llama.cpp
• you run MoE models with experts offloaded to RAM
• you have AVX2+ CPUs or Turing+ NVIDIA GPUs

When to avoid

• you rely on ROCm, Vulkan, or Metal backends, which are not well supported
• you need broad hardware compatibility or the latest upstream llama.cpp features
• you want a project with guaranteed long-term upstream sync

Facets

library · maturity active

llm-inference gpu-computing concurrency large-language-models machine-learning performance windows cpp cli quantization cpu-inference moe llama-cpp-fork ggml linux macos cuda

Member repositories

For agents

markdown · JSON · MCP: product_card(name="ikawrakow/ik_llama.cpp")