# mit-han-lab/smoothquant

[ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

Repository: https://github.com/mit-han-lab/smoothquant
Canonical: https://ross.abutalabs.com/products/smoothquant
Homepage: https://arxiv.org/abs/2211.10438
Language: Python
License: MIT
License Family: permissive
Last push: 2024-07-12T03:11:08+00:00

## Health v2 (maintenance only)
Score: 32/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 0, release rhythm 35, longevity 98
- inputs: {"age_days": 1385, "days_push": 782, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1677, forks 206 (observed 2026-08-28T04:05:21.038115+00:00)

## What it is
SmoothQuant is a training-free post-training quantization (PTQ) method that enables W8A8 (8-bit weight, 8-bit activation) quantization for large language models by migrating quantization difficulty from activations to weights. This repository provides the reference Python implementation, supporting models like OPT, BLOOM, GLM, Llama, Falcon, Mistral, and Mixtral with up to 1.56x speedup and 2x memory reduction.

## Use cases
- quantize an LLM to int8 weights and activations without retraining
- reduce GPU memory usage for serving large language models
- speed up LLM inference with W8A8 quantization
- serve a 530B parameter model on a single node
- evaluate quantization accuracy on Llama, Falcon, Mistral, or Mixtral models
- integrate activation-smoothing quantization into an inference engine

## When to choose
- you need training-free W8A8 quantization of large language models with minimal accuracy loss
- you want to reduce memory footprint and hardware costs for LLM serving
- you are researching or benchmarking post-training quantization methods for LLMs

## When to avoid
- you need 4-bit or lower-bit quantization rather than 8-bit
- you require a production-maintained integration - use TensorRT-LLM, ONNX Runtime, or Neural-Compressor which have adopted SmoothQuant
- you are quantizing small models where activation outliers are not a problem

## Facets
- artifact type: library
- maturity: maintenance
- function: llm-inference, machine-learning, deep-learning
- domain: large-language-models, machine-learning, deep-learning
- platform: python
- tags: quantization, post-training-quantization, w8a8, int8, model-compression, inference-optimization, research-paper, gpu, linux

## Member repositories
- mit-han-lab/smoothquant (main) score 32

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:21.038115+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T03:41:37.128036+00:00, confidence not recorded.
  - readme: https://github.com/mit-han-lab/smoothquant (fetched 2026-08-28T04:05:21.038115+00:00, sha 6af10e2a4a7f)
  - homepage: https://arxiv.org/abs/2211.10438 (fetched 2026-08-29T11:15:03.223160+00:00, sha 4ea8851b899e)
  - site_page: https://info.arxiv.org/about/donate.html (fetched 2026-08-29T11:15:03.225801+00:00, sha cca9c3a11c56)
  - site_page: https://info.arxiv.org/about/ourmembers.html (fetched 2026-08-29T11:15:03.229102+00:00, sha 47cbc55ff1de)
  - site_page: https://info.arxiv.org/about (fetched 2026-08-29T11:15:03.230888+00:00, sha a1f16f915a9a)
  - site_page: https://info.arxiv.org/labs/index.html (fetched 2026-08-29T11:15:03.227544+00:00, sha b14a8d05a0ec)
- Data as of 2026-08-30T08:39:29.467469+00:00.