EverMind-AI/MSA
Memory Sparse Attention - A scalable, end-to-end trainable latent-memory framework for 100M-token contexts. observed · 2026-08-28
Health v2 · maintenance only
53/100
- Activity 81
- Release rhythm 35
- Longevity 22
Flags: no_releases no_license
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-09-02. Adoption (stars, forks) is never an input.
- gap_med: n/a
- age_days: 308
- days_rel: n/a
- days_push: 119
- n_releases_24m: 0
Adoption not part of the score
3515 stars · 228 forks observed · 2026-08-28
What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded
MSA (Memory Sparse Attention) is a Python framework for end-to-end trainable sparse latent-memory attention that scales LLM context to 100M tokens with near-linear complexity. It combines document-wise RoPE, KV cache compression, and a Memory Parallel inference engine to run 100M-token inference on 2×A800 GPUs.
Use cases
- process 100M-token contexts with an LLM
- run needle-in-a-haystack long-context QA
- replace RAG with an end-to-end trainable memory model
- compress KV cache for long-context inference
- summarize entire large corpora with an LLM
- build agents with lifetime-scale conversation memory
When to choose
- you need extreme long-context (tens of millions of tokens) with minimal degradation
- you want an alternative to RAG pipelines with end-to-end differentiability
- you have multi-GPU hardware and need near-linear training/inference complexity
When to avoid
- you only need standard 128K–1M context lengths
- you need a production-ready solution with broad ecosystem support
- you require a permissively licensed dependency and license clarity matters (no license file in repo)
Facets
framework · maturity active
machine-learning llm-inference llm-training rag search-engine large-language-models deep-learning artificial-intelligence python sparse-attention long-context kv-cache-compression latent-memory transformers research retrieval-augmented-generation gpu linux
6 sources
- readme: https://github.com/EverMind-AI/MSA · fetched 2026-08-28 · f74022b69be0
- homepage: https://arxiv.org/abs/2603.23516 · fetched 2026-08-29 · b1c8c8ed970b
- site_page: https://info.arxiv.org/about/donate.html · fetched 2026-08-29 · cca9c3a11c56
- site_page: https://info.arxiv.org/about/ourmembers.html · fetched 2026-08-29 · 47cbc55ff1de
- site_page: https://info.arxiv.org/about · fetched 2026-08-29 · a1f16f915a9a
- site_page: https://info.arxiv.org/labs/index.html · fetched 2026-08-29 · b14a8d05a0ec
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| EverMind-AI/MSA | main | 53 |
For agents
Data as of 2026-08-30T08:39:29.467469+00:00 · Report a problem