# lucidrains/performer-pytorch

An implementation of Performer, a linear attention-based transformer, in Pytorch

Repository: https://github.com/lucidrains/performer-pytorch
Canonical: https://ross.abutalabs.com/products/performer-pytorch
Language: Python
License: MIT
License Family: permissive
Topics: artificial-intelligence, deep-learning, attention-mechanism, attention, transformers
Last push: 2022-02-02T20:33:32+00:00

## Health v2 (maintenance only)
Score: 23/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 0, release rhythm 8, longevity 100
- inputs: {"age_days": 2160, "days_push": 1673, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1181, forks 148 (observed 2026-08-28T04:03:54.073361+00:00)

## What it is
A PyTorch implementation of the Performer transformer, which uses linear attention via the FAVOR+ (Fast Attention Via positive Orthogonal Random features) method. It provides both a standalone Performer layer and a full PerformerLM language model for scalable sequence modeling.

## Use cases
- train a transformer with linear attention for long sequences
- implement FAVOR+ kernel attention in PyTorch
- build an autoregressive language model that scales beyond quadratic attention
- experiment with efficient attention variants for research
- apply transformer attention to images or other modalities
- reduce memory usage of self-attention on long inputs

## When to choose
- you need linear-scaling attention for long sequences in PyTorch
- you want to reproduce or build on the Performer paper
- you want a ready-made language model with efficient attention

## When to avoid
- you need a production-grade, actively maintained transformer library
- you want standard softmax attention with mature ecosystem support
- you work outside PyTorch

## Facets
- artifact type: library
- maturity: maintenance
- function: machine-learning, deep-learning
- domain: deep-learning, large-language-models, artificial-intelligence
- platform: python
- tags: transformers, attention-mechanism, linear-attention, favor-plus, pytorch, efficient-attention

## Member repositories
- lucidrains/performer-pytorch (main) score 23

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:03:54.073361+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T06:25:12.173929+00:00, confidence not recorded.
  - readme: https://github.com/lucidrains/performer-pytorch (fetched 2026-08-28T04:03:54.073361+00:00, sha 9141e9b86dae)
  - registry_pypi: https://pypi.org/pypi/performer-pytorch/json (fetched 2026-08-29T12:32:17.472282+00:00, sha 4e9dc336e53c)
- Data as of 2026-08-30T08:39:29.467469+00:00.