feyninc/chonkie

Health v2 · maintenance only

Adoption not part of the score

4704 stars · 351 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Chonkie is a lightweight Python library for chunking text in RAG pipelines, offering multiple chunking algorithms, embeddings, and integrations with vector databases. It handles the ingestion step of retrieval-augmented generation: fetch, chunk, refine, embed, and store text.

Use cases

• split documents into chunks for a RAG pipeline
• chunk text semantically before embedding
• prepare documents for a vector database
• replace a custom text splitter with a maintained library
• build a fast lightweight document ingestion pipeline
• chunk multilingual text for retrieval

When to choose

• you need efficient, fast text chunking for RAG without heavy dependencies
• you want built-in integrations with tokenizers, embedders, and vector DBs
• you need multilingual chunking support out of the box

When to avoid

• you need a full RAG framework with retrieval and generation, not just ingestion
• your project is not Python-based and you cannot use the JS SDK or API

Facets

library · maturity active

rag nlp etl machine-learning large-language-models artificial-intelligence python cross-platform chunking text-splitter ingestion vector-databases embeddings semantic-chunking retrieval-augmented-generation natural-language-processing

Member repositories

For agents

markdown · JSON · MCP: product_card(name="feyninc/chonkie")