openai/sparse_attention

Health v2 · maintenance only

Flags: no_releases archived no_license

Adoption not part of the score

1613 stars · 192 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A Python library of sparse attention primitives from OpenAI's Sparse Transformers paper, including fused normal attention and strided/fixed block-sparse attention kernels built on the blocksparse package. It also provides a recompute decorator for memory-efficient attention.

Use cases

• implement sparse transformers for long sequences
• speed up attention computation on GPUs
• use strided and fixed attention patterns
• reduce memory usage in transformer training with recomputation
• experiment with block-sparse attention kernels
• model long documents or sequences efficiently

When to choose

• you need block-sparse attention kernels matching the Sparse Transformers paper
• you work with TensorFlow and NVIDIA GPUs including Tensor Cores
• you want reference implementations of strided and fixed attention

When to avoid

• you need maintained or updated code - the repository is archived
• you use PyTorch rather than TensorFlow
• you lack an NVIDIA GPU with the required compute capability
• you want modern efficient attention like FlashAttention

Facets

library · maturity abandoned

machine-learning deep-learning gpu-computing deep-learning large-language-models gpu-computing python sparse-attention transformers cuda-kernels blocksparse archived gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="openai/sparse_attention")