zilliztech/GPTCache

Health v2 · maintenance only

Adoption not part of the score

8172 stars · 593 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

GPTCache is a Python library for building a semantic cache that stores and retrieves LLM responses based on embedding similarity, reducing API costs and latency. It integrates with LangChain and llama_index and offers a Docker server usable from any language.

Use cases

• cache LLM API responses to cut costs
• speed up chatbot responses with semantic caching
• reduce duplicate OpenAI API calls
• add a memcache layer for AIGC applications
• integrate semantic caching into LangChain pipelines

When to choose

• your LLM app has high traffic with repeated or similar queries
• you want to lower OpenAI/LLM API bills and latency
• you use LangChain or llama_index and want drop-in caching

When to avoid

• you need support for brand-new LLM APIs (no longer added)
• you need a stable, unchanging API (it may change)
• your queries are mostly unique with little semantic overlap

Facets

library · maturity maintenance

caching llm-inference rag search-engine large-language-models artificial-intelligence chatbots databases python self-hosted semantic-cache llm-cost-reduction vector-search langchain llama-index openai milvus redis docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="zilliztech/GPTCache")