# thu-ml/SageAttention

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

Repository: https://github.com/thu-ml/SageAttention
Canonical: https://ross.abutalabs.com/products/sageattention
Homepage: https://arxiv.org/abs/2410.02367
Language: Cuda
License: Apache-2.0
License Family: permissive
Topics: attention, inference-acceleration, llm, quantization, cuda, triton, video-generation, efficient-attention, mlsys, llm-infra, vit, video-generate
Last push: 2026-01-17T17:43:28+00:00

## Health v2 (maintenance only)
Score: 40/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 62, release rhythm 8, longevity 49
- inputs: {"age_days": 699, "days_push": 228, "days_rel": 582, "gap_med": null, "n_releases_24m": 1}
- flags: prerelease_only
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 3684, forks 491 (observed 2026-08-28T04:08:13.822599+00:00)

## What it is
SageAttention is a family of quantized attention kernels (INT8/FP8/FP4) that accelerate transformer inference 2-5x over FlashAttention with negligible accuracy loss. It works plug-and-play across language, image, and video generation models on Ampere, Ada, Hopper, and Blackwell GPUs.

## Use cases
- speed up LLM inference attention computation
- accelerate video generation model inference
- quantize attention to int8 or fp8 for faster transformers
- replace flashattention with a faster drop-in kernel
- reduce inference latency for diffusion transformer models
- run ViT models faster on consumer GPUs

## When to choose
- you need faster transformer inference on NVIDIA GPUs without retraining
- you run LLM, image, or video generation models and attention is the bottleneck
- you want plug-and-play acceleration compatible with torch.compile and distributed inference

## When to avoid
- you need bit-exact attention outputs for precision-sensitive training or evaluation
- you target non-NVIDIA hardware like AMD or Apple GPUs
- your model uses attention variants unsupported by the kernels

## Facets
- artifact type: library
- maturity: active
- function: llm-inference, machine-learning, gpu-computing, deep-learning
- domain: large-language-models, machine-learning, gpu-computing, image-processing
- platform: python, windows
- tags: quantization, attention-kernels, flashattention, inference-acceleration, int8, fp8, fp4, triton, video-generation, diffusion-models, plug-and-play, video, gpu, cuda, linux

## Member repositories
- thu-ml/SageAttention (main) score 40

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:08:13.822599+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T18:31:11.557830+00:00, confidence not recorded.
  - readme: https://github.com/thu-ml/SageAttention (fetched 2026-08-28T04:08:13.822599+00:00, sha aa6226e339e1)
  - homepage: https://arxiv.org/abs/2410.02367 (fetched 2026-08-29T09:25:01.917440+00:00, sha da51c7a1a436)
  - site_page: https://info.arxiv.org/about/donate.html (fetched 2026-08-29T09:25:01.919885+00:00, sha cca9c3a11c56)
  - site_page: https://info.arxiv.org/about/ourmembers.html (fetched 2026-08-29T09:25:01.922988+00:00, sha 47cbc55ff1de)
  - site_page: https://info.arxiv.org/about (fetched 2026-08-29T09:25:01.925053+00:00, sha a1f16f915a9a)
  - site_page: https://info.arxiv.org/labs/index.html (fetched 2026-08-29T09:25:01.921453+00:00, sha b14a8d05a0ec)
- Data as of 2026-08-30T08:39:29.467469+00:00.