# pytorch/ao

PyTorch native quantization for training and inference

Repository: https://github.com/pytorch/ao
Canonical: https://ross.abutalabs.com/products/ao
Homepage: https://pytorch.org/ao/stable/index.html
Language: Python
License: NOASSERTION
License Family: other
Topics: brrr, dtypes, inference, mx, pytorch, quantization, sparsity, training, float8, transformer, cuda, llama
Last push: 2026-08-26T15:25:08+00:00

## Health v2 (maintenance only)
Score: 89/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 84, longevity 73
- inputs: {"age_days": 1034, "days_push": 7, "days_rel": 30, "gap_med": 46, "n_releases_24m": 14}
- flags: no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2957, forks 604 (observed 2026-08-28T04:07:32.242631+00:00)

## What it is
TorchAO is a PyTorch-native library for model optimization through quantization and sparsity. It supports quantizing weights, gradients, optimizers, and activations using custom data types like int4, float8, and MX formats for both training and inference.

## Use cases
- quantize llama models to int4 for faster inference
- speed up llm pre-training with float8
- apply quantization-aware training to recover accuracy
- reduce model memory footprint for serving
- sparsify model weights and gradients
- quantize models for vllm or huggingface serving

## When to choose
- you use PyTorch and want native quantization for training or inference
- you need int4/float8/MX data types for LLM speedups
- you want QAT to recover quantization accuracy loss
- you serve models with vLLM, HF Transformers, or torchtune

## When to avoid
- you need a framework-agnostic quantizer outside the PyTorch ecosystem
- you target CPU-only inference with no CUDA/XPU support
- you need post-training quantization for non-PyTorch models like ONNX

## Facets
- artifact type: library
- maturity: active
- function: machine-learning, llm-inference, llm-training, gpu-computing
- domain: deep-learning, large-language-models, machine-learning, gpu-computing
- platform: python
- tags: quantization, sparsity, float8, int4, qat, model-optimization, pytorch, inference-optimization, training-optimization, gpu, cuda, linux

## Member repositories
- pytorch/ao (main) score 89

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:07:32.242631+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T18:47:32.080510+00:00, confidence not recorded.
  - readme: https://github.com/pytorch/ao (fetched 2026-08-28T04:07:32.242631+00:00, sha b389888aa48b)
  - homepage: https://pytorch.org/ao/stable/index.html (fetched 2026-08-29T09:47:34.722334+00:00, sha 00b8e66fa183)
  - site_page: https://docs.pytorch.org/docs/stable/index.html (fetched 2026-08-29T09:47:34.731242+00:00, sha b3da68f8848c)
- Data as of 2026-08-30T08:39:29.467469+00:00.