intel/auto-round

Health v2 · maintenance only

Adoption not part of the score

1588 stars · 168 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

AutoRound is Intel's advanced quantization toolkit for LLMs and vision-language models, using sign-gradient descent to achieve high accuracy at 2-4 bit widths. It supports multiple datatypes and hardware (CPU, XPU, CUDA) and integrates with vLLM, SGLang, and Hugging Face Transformers.

Use cases

• quantize an LLM to int4 for faster inference
• compress a large language model to run on CPU
• convert models to gguf or mxfp4 formats
• quantize a vision-language model with minimal accuracy loss
• prepare quantized models for vLLM or SGLang deployment
• calibration-free low-bit quantization of open LLMs

When to choose

• you need high-accuracy 2-4 bit weight-only quantization for LLMs or VLMs
• you deploy on Intel CPU/GPU or NVIDIA CUDA and want vLLM/SGLang compatibility
• you want multiple output formats like GGUF, MXFP4, or NVFP4 from one toolkit

When to avoid

• you need quantization for non-transformer model architectures
• you want training-time or QAT quantization rather than post-training weight-only
• you require a fully stable, long-term-supported API given rapid experimental changes

Facets

library · maturity active

llm-inference machine-learning cli sdk large-language-models machine-learning deep-learning developer-tools python windows quantization int4 low-bit-quantization signround vllm sglang gguf mxfp4 nvfp4 vision-language-models weight-only-quantization model-compression linux macos gpu cpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="intel/auto-round")