lucidrains/reformer-pytorch

Health v2 · maintenance only

Adoption not part of the score

2191 stars · 253 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A PyTorch implementation of the Reformer, an efficient Transformer architecture using LSH attention, reversible networks, and chunking to handle very long sequences. It provides a ReformerLM language model validated on auto-regressive tasks like enwik8.

Use cases

• train a transformer on very long sequences with limited memory
• implement LSH attention in pytorch
• build a language model that handles 32k+ token contexts
• experiment with reversible networks and chunked feedforward layers
• run auto-regressive language modeling on enwik8

When to choose

• you need memory-efficient attention for extremely long sequences in PyTorch
• you want to reproduce or experiment with the Reformer paper
• you need reversible networks and factorized embeddings to cut memory usage

When to avoid

• you need a general-purpose, actively maintained transformer library
• you want standard full attention for short sequences
• you need production LLM inference rather than research experimentation

Facets

library · maturity maintenance

machine-learning deep-learning llm-training deep-learning large-language-models machine-learning python transformer lsh-attention reformer pytorch efficient-attention long-context gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="lucidrains/reformer-pytorch")