hhhuang/CAG

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1607 stars · 228 forks observed · 2026-10-16

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Cache-Augmented Generation (CAG) is a Python implementation of a retrieval-free alternative to RAG that preloads knowledge into an LLM's context and caches the KV-cache for direct inference. It accompanies an ACM Web Conference 2025 short paper and includes experiment scripts comparing CAG against traditional RAG.

Use cases

• avoid retrieval latency when answering questions over a small knowledge base
• replace RAG pipeline with preloaded context and KV-cache
• run question answering experiments on SQuAD and HotpotQA
• benchmark LLM performance vs context length
• reduce retrieval errors in LLM applications

When to choose

• your knowledge source fits within the LLM context window
• you want lower latency and simpler architecture than RAG
• you are researching retrieval-free generation approaches

When to avoid

• your knowledge base is too large for the context window
• you need frequently updated knowledge without re-caching
• you require a production-ready retrieval system

Facets

library · maturity active

rag llm-inference caching machine-learning large-language-models artificial-intelligence python cache-augmented-generation kv-cache retrieval-free llm research-paper retrieval-augmented-generation natural-language-processing

Member repositories

For agents

markdown · JSON · MCP: product_card(name="hhhuang/CAG")