NVIDIA/kvpress

Health v2 · maintenance only

Adoption not part of the score

1211 stars · 180 forks observed · 2026-09-23

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

kvpress is a Python library from NVIDIA that implements multiple KV cache compression methods and benchmarks for long-context LLM inference, built on Hugging Face transformers. It provides 'presses' that compress the key-value cache during prefilling to reduce memory usage, and simplifies developing new compression methods.

Use cases

• compress kv cache for long-context llm inference
• reduce gpu memory when serving llms with long prompts
• benchmark kv cache compression methods
• implement a custom kv cache compression method
• run 1M token contexts on limited gpu memory
• integrate kv cache compression into a transformers pipeline

When to choose

• you deploy or fine-tune transformer LLMs with long contexts and hit KV cache memory limits
• you research KV cache compression and want a benchmarking framework with existing methods
• you already use Hugging Face transformers and PyTorch

When to avoid

• you need KV cache compression for non-transformer or non-Hugging Face model runtimes
• you want a production inference server rather than a compression library
• your contexts are short enough that KV cache memory is not a bottleneck

Facets

library · maturity active

llm-inference machine-learning benchmarking large-language-models deep-learning machine-learning python cross-platform kv-cache kv-cache-compression long-context transformers pytorch huggingface memory-optimization gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="NVIDIA/kvpress")