vllm-project/llm-compressor

Health v2 · maintenance only

Adoption not part of the score

3726 stars · 637 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

LLM Compressor is a Python library for applying quantization and pruning algorithms to large language models, producing compressed-tensors checkpoints optimized for vLLM deployment. It integrates with Hugging Face Transformers and supports weight, activation, KV cache, and attention quantization including MoE expert pruning.

Use cases

• quantize an LLM to fp8 or int4 for vLLM inference
• reduce GPU memory requirements for serving large language models
• compress a Hugging Face model into compressed-tensors format
• prune experts from a mixture-of-experts model
• speed up LLM inference with lower-precision weights

When to choose

• you deploy models with vLLM and want smaller, faster checkpoints
• you need state-of-the-art quantization schemes like NVFP4, FP8, or arbitrary bit-widths
• you work with Hugging Face Transformers models and want a pip-installable optimization pipeline

When to avoid

• you expect faster inference through plain transformers - compressed models only accelerate in vLLM or compatible servers
• you need a serving engine itself rather than a model compression tool
• you require SGLang support, which is minimally maintained

Facets

library · maturity active

machine-learning llm-training compression serialization large-language-models machine-learning deep-learning gpu-computing python cross-platform quantization model-optimization vllm compressed-tensors hugging-face moe pruning fp8 int4 linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="vllm-project/llm-compressor")