# ikawrakow/ik_llama.cpp

llama.cpp fork with additional SOTA quants and improved performance

Repository: https://github.com/ikawrakow/ik_llama.cpp
Canonical: https://ross.abutalabs.com/products/ik_llamacpp
Language: C++
License: MIT
License Family: permissive
Last push: 2026-08-26T17:54:12+00:00

## Health v2 (maintenance only)
Score: 59/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 8, longevity 56
- inputs: {"age_days": 797, "days_push": 7, "days_rel": 407, "gap_med": null, "n_releases_24m": 1}
- flags: prerelease_only
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 3091, forks 431 (observed 2026-08-28T04:07:42.273826+00:00)

## What it is
A fork of llama.cpp offering state-of-the-art quantization types and improved CPU/CUDA inference performance for large language models. It pioneered features like MLA, quant repacking, and tensor parallelism before they landed upstream.

## Use cases
- run LLMs locally with faster CPU inference
- quantize models with better quality-per-bit formats
- run MoE models with experts on CPU and attention on GPU
- get better prompt processing speed than mainline llama.cpp
- run DeepSeek-style MLA models efficiently
- serve LLMs on consumer hardware with limited VRAM

## When to choose
- you want maximum CPU or hybrid CPU/GPU inference performance for LLMs
- you need advanced quantization types not in mainline llama.cpp
- you run MoE models with experts offloaded to RAM
- you have AVX2+ CPUs or Turing+ NVIDIA GPUs

## When to avoid
- you rely on ROCm, Vulkan, or Metal backends, which are not well supported
- you need broad hardware compatibility or the latest upstream llama.cpp features
- you want a project with guaranteed long-term upstream sync

## Facets
- artifact type: library
- maturity: active
- function: llm-inference, gpu-computing, concurrency
- domain: large-language-models, machine-learning, performance
- platform: windows, cpp, cli
- tags: quantization, cpu-inference, moe, llama-cpp-fork, ggml, linux, macos, cuda

## Member repositories
- ikawrakow/ik_llama.cpp (main) score 59

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:07:42.273826+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T07:27:20.869052+00:00, confidence not recorded.
  - readme: https://github.com/ikawrakow/ik_llama.cpp (fetched 2026-08-28T04:07:42.273826+00:00, sha 19b4c4a0a691)
- Data as of 2026-08-30T08:39:29.467469+00:00.