abertsch72/unlimiformer

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1062 stars · 78 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Unlimiformer is a method and official implementation for augmenting pretrained encoder-decoder transformers with retrieval-based attention, enabling unlimited-length inputs without changing the mathematical definition of attention. It supports models like BART and Llama-2 for tasks such as long-document summarization and prompting with entire books.

Use cases

• process unlimited length inputs with pretrained encoder-decoder transformers
• summarize entire books with Llama-2
• improve long-range attention in seq2seq models without retraining from scratch
• apply retrieval-based attention over long documents
• train models with Unlimiformer for long-input tasks

When to choose

• you need to feed very long documents to a pretrained encoder-decoder model like BART
• you want to prompt Llama-2 with inputs longer than its context window
• you want retrieval-based attention without changing attention semantics

When to avoid

• you need a simple short-context model with no long-input requirements
• you cannot afford the extra memory/compute for kNN datastores and indexes
• you use decoder-only models other than supported Llama derivatives

Facets

library · maturity stable

machine-learning deep-learning search-engine llm-inference large-language-models deep-learning machine-learning python long-context retrieval-based-attention transformers encoder-decoder neurips-2023 llama-2 knn-attention natural-language-processing

Member repositories

For agents

markdown · JSON · MCP: product_card(name="abertsch72/unlimiformer")