scrya-com/rotorquant

Health v2 · maintenance only

Flags: no_releases young no_license

Adoption not part of the score

1043 stars · 89 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

RotorQuant is a KV cache compression method for LLM inference that replaces full d×d orthogonal rotations with block-diagonal Clifford rotor transforms, integrated as a drop-in quantization scheme in llama.cpp. It claims better perplexity and substantially faster decode/prefill than TurboQuant at the same compression ratio.

Use cases

• compress kv cache for long-context llm inference
• speed up llama.cpp decode and prefill
• reduce gpu memory for llm serving
• quantize attention k/v vectors at 3-bit
• compare against turboquant baselines

When to choose

• you serve LLMs via llama.cpp and need lower memory or higher throughput at low bit-widths
• you want KV cache quantization with minimal quality loss (PPL) and fast kernels on CUDA or Metal

When to avoid

• you need a permissively licensed dependency - the repo has no license
• you need a framework-agnostic solution outside llama.cpp/Triton
• you require independently verified benchmarks - results come from the authors' own paper

Facets

library · maturity active

llm-inference caching gpu-computing benchmarking large-language-models machine-learning performance gpu-computing python cpp windows kv-cache quantization clifford-algebra llama-cpp cuda metal inference-optimization triton gpu linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="scrya-com/rotorquant")