# microsoft/LLMLingua

[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

Repository: https://github.com/microsoft/LLMLingua
Canonical: https://ross.abutalabs.com/products/llmlingua
Homepage: https://llmlingua.com/
Language: Python
License: MIT
License Family: permissive
Last push: 2026-04-08T18:59:13+00:00

## Health v2 (maintenance only)
Score: 53/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 76, release rhythm 8, longevity 82
- inputs: {"age_days": 1153, "days_push": 147, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 6609, forks 418 (observed 2026-08-28T04:09:45.697865+00:00)

## What it is
LLMLingua is a Microsoft library for prompt compression that removes non-essential tokens from prompts and KV caches to speed up LLM inference, achieving up to 20x compression with minimal performance loss. It includes variants like LongLLMLingua for query-aware long-context compression and LLMLingua-2 for faster task-agnostic compression, and integrates with LangChain, LlamaIndex, and Prompt flow.

## Use cases
- compress prompts before sending to an LLM to cut token costs
- speed up LLM inference by shrinking long context windows
- reduce RAG context size without losing answer quality
- compress long documents for query-aware question answering
- lower KV cache memory usage for long-context models
- make prompts fit within a smaller context limit

## When to choose
- you pay per token and want to cut prompt costs
- your RAG pipeline stuffs too much retrieved context into prompts
- long-context inference latency is a bottleneck
- you use LangChain or LlamaIndex and want drop-in compression

## When to avoid
- your prompts are already short and cheap
- your task is highly sensitive to dropped tokens, such as precise code generation
- you need compression without any small auxiliary model or extra compute
- you work outside the Python ecosystem

## Facets
- artifact type: library
- maturity: active
- function: llm-inference, rag, nlp, machine-learning
- domain: large-language-models, artificial-intelligence
- platform: python
- tags: prompt-compression, kv-cache, inference-acceleration, token-reduction, cost-optimization, natural-language-processing, retrieval-augmented-generation

## Member repositories
- microsoft/LLMLingua (main) score 53

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:09:45.697865+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T17:43:31.029996+00:00, confidence not recorded.
  - readme: https://github.com/microsoft/LLMLingua (fetched 2026-08-28T04:09:45.697865+00:00, sha 19f01f05192d)
  - homepage: https://llmlingua.com/ (fetched 2026-08-29T08:39:45.937669+00:00, sha 5ad6a049c9bf)
  - registry_pypi: https://pypi.org/pypi/llmlingua/json (fetched 2026-08-29T08:39:45.947852+00:00, sha f044ec05f3db)
- Data as of 2026-08-30T08:39:29.467469+00:00.