Jamie-Stirling/RetNet

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1209 stars · 106 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A minimal, pure PyTorch implementation of the Retentive Network (RetNet) architecture proposed as a successor to Transformers for large language models. It implements parallel, recurrent, and chunkwise retention paradigms plus a causal language model built on top.

Use cases

• implement retnet architecture in pytorch
• experiment with retention network instead of transformer
• train a causal language model with retnet
• compare parallel recurrent and chunkwise retention paradigms
• study the retnet paper implementation
• research alternative attention-free sequence models

When to choose

• you want a readable, correctness-focused reference implementation of RetNet
• you are researching retention-based sequence models in PyTorch
• you need all three retention paradigms (parallel, recurrent, chunkwise)

When to avoid

• you need a production-optimized, high-performance training stack
• you require half-precision complex positional encodings
• you want a maintained framework with releases and long-term support

Facets

library · maturity experimental

machine-learning deep-learning llm-training deep-learning large-language-models python retnet pytorch transformer-alternative research-implementation retention-mechanism natural-language-processing

Member repositories

For agents

markdown · JSON · MCP: product_card(name="Jamie-Stirling/RetNet")