# fla-org/flash-linear-attention

🚀 Efficient implementations for emerging model architectures

Repository: https://github.com/fla-org/flash-linear-attention
Canonical: https://ross.abutalabs.com/products/flash-linear-attention
Homepage: https://github.com/fla-org/flash-linear-attention
Language: Python
License: MIT
License Family: permissive
Topics: large-language-models, machine-learning-systems, natural-language-processing, sequence-modeling
Last push: 2026-08-26T12:15:05+00:00

## Health v2 (maintenance only)
Score: 88/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 83, longevity 70
- inputs: {"age_days": 987, "days_push": 7, "days_rel": 37, "gap_med": 39.5, "n_releases_24m": 15}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 5627, forks 672 (observed 2026-08-28T04:09:25.802639+00:00)

## What it is
A PyTorch library providing hardware-efficient implementations of emerging sequence model architectures, including linear attention, sparse attention, state space models (Mamba variants), and hybrid LLM layers. Kernels are written in Triton and related backends and verified on NVIDIA, AMD, and Intel GPUs.

## Use cases
- implement linear attention layers for training LLMs
- run Mamba and state space model layers efficiently on GPU
- train hybrid attention and SSM language models
- use gated delta net kernels in my model
- benchmark efficient attention implementations across NVIDIA AMD and Intel GPUs
- build constant-memory sequence models for long context
- try emerging token mixing architectures like KDA and GDN-2

## When to choose
- you need production-quality Triton kernels for linear attention or SSM layers
- you are training or fine-tuning LLMs with non-transformer or hybrid token mixing
- you want a single library covering many recent sequence-model papers
- you need multi-vendor GPU support (NVIDIA, AMD, Intel)

## When to avoid
- you only need standard softmax attention with no custom layers
- you work outside PyTorch or on CPU-only hardware
- you need a turnkey chatbot or inference server rather than model layers
- your project depends on fully stable, long-term-frozen APIs

## Facets
- artifact type: library
- maturity: active
- function: machine-learning, llm-training, llm-inference, gpu-computing, deep-learning
- domain: large-language-models, deep-learning, machine-learning, gpu-computing
- platform: python, cross-platform
- tags: linear-attention, state-space-models, triton-kernels, sequence-modeling, mamba, gated-deltanet, sparse-attention, hybrid-architectures, pytorch, natural-language-processing, gpu, linux

## Member repositories
- fla-org/flash-linear-attention (main) score 88

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:09:25.802639+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T17:55:21.398178+00:00, confidence not recorded.
  - readme: https://github.com/fla-org/flash-linear-attention (fetched 2026-08-28T04:09:25.802639+00:00, sha 90d21d467d3b)
  - homepage: https://github.com/fla-org/flash-linear-attention (fetched 2026-08-29T08:49:57.721608+00:00, sha 720ba67b2559)
  - registry_pypi: https://pypi.org/pypi/flash-linear-attention/json (fetched 2026-08-29T08:49:57.731032+00:00, sha 6745bc30119f)
- Data as of 2026-08-30T08:39:29.467469+00:00.