cocoindex-io/cocoindex

Health v2 · maintenance only

Adoption not part of the score

11409 stars · 888 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

CocoIndex is an open-source incremental data framework for AI, with a Rust engine and a declarative Python API that keeps sources (files, S3, Postgres, Kafka, etc.) in sync with targets like pgvector, Qdrant, Neo4j, and LanceDB by reprocessing only the delta. It is aimed at building continuously fresh context for AI agents, including RAG pipelines, semantic search, knowledge graphs, and real-time codebase indexing.

Use cases

• build a rag pipeline over my documents
• keep a vector index of my codebase in sync with every commit
• incrementally embed files into postgres pgvector
• build a knowledge graph from meeting notes or docs
• semantic search over pdfs and markdown
• feed fresh context to coding and code-review agents
• stream csv or kafka data into a vector database
• index code call graphs and symbols for an ai agent

When to choose

• you need continuously updated indexes where reprocessing everything on each change is too slow or expensive
• you want a declarative python flow from sources like s3, postgres, or local files to vector stores or graph databases
• you are building ai agents that need fresh retrieval context, including codebase intelligence
• you want incremental etl with built-in connectors and only-delta processing

When to avoid

• you need a one-off batch etl job with no ongoing sync
• your pipeline is simple enough that a plain script or existing etl tool suffices
• you need a target store or source not covered by its connectors and don't want to write a custom connector
• you require a mature v1 ecosystem — the v0 docs are already in maintenance mode

Facets

framework · maturity active

etl rag vector-database search-engine data-science llm-inference agent-framework streaming workflow-automation large-language-models databases developer-tools python rust cross-platform self-hosted incremental-processing change-data-capture semantic-search knowledge-graph vector-embeddings codebase-indexing declarative-pipelines context-engineering live-indexing pgvector data-engineering retrieval-augmented-generation ai-agents search docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="cocoindex-io/cocoindex")