meta-pytorch/attention-gym

Health v2 · maintenance only

Adoption not part of the score

1234 stars · 79 forks observed · 2026-09-03

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Attention Gym is a collection of tools, examples, and reference implementations for working with PyTorch's FlexAttention API. It provides attention mask and score-mod utilities, attention variant implementations, and performance comparisons for researchers and developers.

Use cases

• experiment with attention mechanisms using flex-attention
• create sliding window or document attention masks in pytorch
• benchmark different attention variants
• learn how to use flex_attention in my model
• reference implementations of linear attention

When to choose

• you use PyTorch 2.5+ FlexAttention and want ready-made mask/score-mod examples
• you are researching or comparing attention mechanisms
• you want utility functions for building and combining block masks

When to avoid

• you need a stable API with backward compatibility guarantees
• you don't use PyTorch or FlexAttention
• you need production-hardened attention kernels rather than examples

Facets

library · maturity active

machine-learning deep-learning developer-tools benchmarking deep-learning machine-learning large-language-models developer-tools python cross-platform flex-attention pytorch attention-mechanisms transformers research gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="meta-pytorch/attention-gym")