deepseek-ai/Engram

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

4614 stars · 357 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Official implementation of Engram, a conditional memory module from DeepSeek that modernizes N-gram embeddings for O(1) lookup as a new sparsity axis for large language models. It complements Mixture-of-Experts by offloading static knowledge into large embedding tables that can be hosted in host memory with minimal inference overhead.

Use cases

• add conditional memory to an LLM architecture
• train a language model with N-gram lookup memory
• reduce inference cost by offloading embedding tables to host memory
• study sparsity allocation between MoE and static memory
• reproduce the Engram-27B paper experiments
• improve knowledge-heavy LLM benchmarks under iso-FLOPs constraints

When to choose

• you are researching sparse memory architectures for LLMs
• you want to complement MoE with a static knowledge lookup module
• you need the official reference implementation of the Engram paper

When to avoid

• you need a production-ready general LLM framework rather than a research module
• you lack GPU infrastructure for large model training
• you want a plug-and-play library with broad model support

Facets

library · maturity active

machine-learning llm-training llm-inference deep-learning large-language-models deep-learning machine-learning artificial-intelligence python n-gram-embeddings conditional-memory mixture-of-experts sparsity lookup-tables research deepseek gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="deepseek-ai/Engram")