casper-hansen/AutoAWQ

Health v2 · maintenance only

Flags: archived

Adoption not part of the score

2349 stars · 307 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

AutoAWQ is a Python library implementing the Activation-aware Weight Quantization (AWQ) algorithm for 4-bit quantization of large language models, offering faster inference and reduced memory usage versus FP16. It is now officially deprecated and has been adopted by the vLLM project's llm-compressor.

Use cases

• quantize llm to 4-bit
• run quantized language models on gpu
• reduce vram usage for llm inference
• convert huggingface models to awq format
• export quantized models to gguf
• run llm inference on amd rocm or cpu

When to choose

• you need to quantize or run existing AWQ 4-bit models with older Torch/Transformers versions
• you want fast 4-bit LLM inference on NVIDIA, AMD, or Intel CPU hardware

When to avoid

• you are starting a new project - use vLLM's llm-compressor or MLX-LM instead
• you need compatibility with the latest Transformers versions
• you expect bug fixes or ongoing maintenance

Facets

library · maturity abandoned

llm-inference machine-learning gpu-computing large-language-models machine-learning deep-learning python windows quantization 4-bit awq huggingface transformers deprecated model-compression gpu linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="casper-hansen/AutoAWQ")