fla-org/flash-linear-attention

Health v2 · maintenance only

Adoption not part of the score

5627 stars · 672 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

A PyTorch library providing hardware-efficient implementations of emerging sequence model architectures, including linear attention, sparse attention, state space models (Mamba variants), and hybrid LLM layers. Kernels are written in Triton and related backends and verified on NVIDIA, AMD, and Intel GPUs.

Use cases

• implement linear attention layers for training LLMs
• run Mamba and state space model layers efficiently on GPU
• train hybrid attention and SSM language models
• use gated delta net kernels in my model
• benchmark efficient attention implementations across NVIDIA AMD and Intel GPUs
• build constant-memory sequence models for long context
• try emerging token mixing architectures like KDA and GDN-2

When to choose

• you need production-quality Triton kernels for linear attention or SSM layers
• you are training or fine-tuning LLMs with non-transformer or hybrid token mixing
• you want a single library covering many recent sequence-model papers
• you need multi-vendor GPU support (NVIDIA, AMD, Intel)

When to avoid

• you only need standard softmax attention with no custom layers
• you work outside PyTorch or on CPU-only hardware
• you need a turnkey chatbot or inference server rather than model layers
• your project depends on fully stable, long-term-frozen APIs

Facets

library · maturity active

machine-learning llm-training llm-inference gpu-computing deep-learning large-language-models deep-learning machine-learning gpu-computing python cross-platform linear-attention state-space-models triton-kernels sequence-modeling mamba gated-deltanet sparse-attention hybrid-architectures pytorch natural-language-processing gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="fla-org/flash-linear-attention")