# EverMind-AI/MSA

Memory Sparse Attention -  A scalable, end-to-end trainable latent-memory framework for 100M-token contexts.

Repository: https://github.com/EverMind-AI/MSA
Canonical: https://ross.abutalabs.com/products/msa
Homepage: https://arxiv.org/abs/2603.23516
Language: Python
License Family: other
Last push: 2026-05-06T09:24:05+00:00

## Health v2 (maintenance only)
Score: 53/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 81, release rhythm 35, longevity 22
- inputs: {"age_days": 308, "days_push": 119, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases, no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 3515, forks 228 (observed 2026-08-28T04:08:07.810662+00:00)

## What it is
MSA (Memory Sparse Attention) is a Python framework for end-to-end trainable sparse latent-memory attention that scales LLM context to 100M tokens with near-linear complexity. It combines document-wise RoPE, KV cache compression, and a Memory Parallel inference engine to run 100M-token inference on 2×A800 GPUs.

## Use cases
- process 100M-token contexts with an LLM
- run needle-in-a-haystack long-context QA
- replace RAG with an end-to-end trainable memory model
- compress KV cache for long-context inference
- summarize entire large corpora with an LLM
- build agents with lifetime-scale conversation memory

## When to choose
- you need extreme long-context (tens of millions of tokens) with minimal degradation
- you want an alternative to RAG pipelines with end-to-end differentiability
- you have multi-GPU hardware and need near-linear training/inference complexity

## When to avoid
- you only need standard 128K–1M context lengths
- you need a production-ready solution with broad ecosystem support
- you require a permissively licensed dependency and license clarity matters (no license file in repo)

## Facets
- artifact type: framework
- maturity: active
- function: machine-learning, llm-inference, llm-training, rag, search-engine
- domain: large-language-models, deep-learning, artificial-intelligence
- platform: python
- tags: sparse-attention, long-context, kv-cache-compression, latent-memory, transformers, research, retrieval-augmented-generation, gpu, linux

## Member repositories
- EverMind-AI/MSA (main) score 53

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:08:07.810662+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T18:35:35.499106+00:00, confidence not recorded.
  - readme: https://github.com/EverMind-AI/MSA (fetched 2026-08-28T04:08:07.810662+00:00, sha f74022b69be0)
  - homepage: https://arxiv.org/abs/2603.23516 (fetched 2026-08-29T09:29:33.585099+00:00, sha b1c8c8ed970b)
  - site_page: https://info.arxiv.org/about/donate.html (fetched 2026-08-29T09:29:33.588103+00:00, sha cca9c3a11c56)
  - site_page: https://info.arxiv.org/about/ourmembers.html (fetched 2026-08-29T09:29:33.591498+00:00, sha 47cbc55ff1de)
  - site_page: https://info.arxiv.org/about (fetched 2026-08-29T09:29:33.593487+00:00, sha a1f16f915a9a)
  - site_page: https://info.arxiv.org/labs/index.html (fetched 2026-08-29T09:29:33.589863+00:00, sha b14a8d05a0ec)
- Data as of 2026-08-30T08:39:29.467469+00:00.