# lucidrains/reformer-pytorch

Reformer, the efficient Transformer, in Pytorch

Repository: https://github.com/lucidrains/reformer-pytorch
Canonical: https://ross.abutalabs.com/products/reformer-pytorch
Language: Python
License: MIT
License Family: permissive
Topics: artificial-intelligence, transformers, attention-mechanism, machine-learning, pytorch
Last push: 2023-06-21T14:17:49+00:00

## Health v2 (maintenance only)
Score: 23/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 0, release rhythm 8, longevity 100
- inputs: {"age_days": 2428, "days_push": 1169, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2191, forks 253 (observed 2026-08-28T04:06:24.134935+00:00)

## What it is
A PyTorch implementation of the Reformer, an efficient Transformer architecture using LSH attention, reversible networks, and chunking to handle very long sequences. It provides a ReformerLM language model validated on auto-regressive tasks like enwik8.

## Use cases
- train a transformer on very long sequences with limited memory
- implement LSH attention in pytorch
- build a language model that handles 32k+ token contexts
- experiment with reversible networks and chunked feedforward layers
- run auto-regressive language modeling on enwik8

## When to choose
- you need memory-efficient attention for extremely long sequences in PyTorch
- you want to reproduce or experiment with the Reformer paper
- you need reversible networks and factorized embeddings to cut memory usage

## When to avoid
- you need a general-purpose, actively maintained transformer library
- you want standard full attention for short sequences
- you need production LLM inference rather than research experimentation

## Facets
- artifact type: library
- maturity: maintenance
- function: machine-learning, deep-learning, llm-training
- domain: deep-learning, large-language-models, machine-learning
- platform: python
- tags: transformer, lsh-attention, reformer, pytorch, efficient-attention, long-context, gpu

## Member repositories
- lucidrains/reformer-pytorch (main) score 23

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:06:24.134935+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T02:47:35.916608+00:00, confidence not recorded.
  - readme: https://github.com/lucidrains/reformer-pytorch (fetched 2026-08-28T04:06:24.134935+00:00, sha ba707ad027b2)
  - registry_pypi: https://pypi.org/pypi/reformer-pytorch/json (fetched 2026-08-29T10:27:49.013641+00:00, sha a9ea7426ee2c)
- Data as of 2026-08-30T08:39:29.467469+00:00.