thu-ml/SpargeAttn

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1033 stars · 102 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

SpargeAttn is the official implementation of a training-free sparse and quantized attention method that accelerates inference for language, image, and video models. It provides plug-and-play CUDA APIs (built on SageAttention2++) that skip matrix multiplications via a two-stage online filter while preserving end-to-end accuracy.

Use cases

• speed up LLM inference with sparse attention
• accelerate video generation model inference
• accelerate diffusion transformer image generation
• drop-in replacement for attention in PyTorch models
• apply custom block-sparse attention masks per head
• balance attention accuracy vs speed with topk sparsity

When to choose

• you need faster attention inference on NVIDIA GPUs without retraining
• you run long-context LLMs or video/image generation models where attention dominates latency
• you want a plug-and-play sparse attention kernel compatible with SageAttention

When to avoid

• you need CPU or non-CUDA hardware support
• you require exact dense attention outputs with no approximation
• your model uses very short sequences where sparsity gains are minimal

Facets

library · maturity active

machine-learning llm-inference gpu-computing sdk deep-learning large-language-models gpu-computing image-processing python sparse-attention quantization inference-acceleration training-free video-generation vision-transformer sageattention cuda-kernel icml2025 video gpu cuda

Member repositories

For agents

markdown · JSON · MCP: product_card(name="thu-ml/SpargeAttn")