lucidrains/performer-pytorch

Health v2 · maintenance only

Adoption not part of the score

1181 stars · 148 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A PyTorch implementation of the Performer transformer, which uses linear attention via the FAVOR+ (Fast Attention Via positive Orthogonal Random features) method. It provides both a standalone Performer layer and a full PerformerLM language model for scalable sequence modeling.

Use cases

• train a transformer with linear attention for long sequences
• implement FAVOR+ kernel attention in PyTorch
• build an autoregressive language model that scales beyond quadratic attention
• experiment with efficient attention variants for research
• apply transformer attention to images or other modalities
• reduce memory usage of self-attention on long inputs

When to choose

• you need linear-scaling attention for long sequences in PyTorch
• you want to reproduce or build on the Performer paper
• you want a ready-made language model with efficient attention

When to avoid

• you need a production-grade, actively maintained transformer library
• you want standard softmax attention with mature ecosystem support
• you work outside PyTorch

Facets

library · maturity maintenance

machine-learning deep-learning deep-learning large-language-models artificial-intelligence python transformers attention-mechanism linear-attention favor-plus pytorch efficient-attention

Member repositories

For agents

markdown · JSON · MCP: product_card(name="lucidrains/performer-pytorch")