ucbepic/docetl

Health v2 · maintenance only

Adoption not part of the score

3995 stars · 427 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

DocETL is a Python library and CLI for building LLM-powered data processing and ETL pipelines over structured and unstructured data using declarative map-reduce operations written in natural language. It automatically optimizes pipelines by swapping models, rewriting prompts, and decomposing operations to improve accuracy and reduce cost.

Use cases

• extract structured fields from unstructured documents with llms
• classify and summarize thousands of support tickets
• build etl pipelines that use llm calls as operators
• process large collections of pdfs or text into queryable tables
• optimize llm data processing pipelines for cost and accuracy
• run map-reduce style analysis over unstructured text data

When to choose

• you need to process large volumes of unstructured text or documents with llms
• you want declarative, optimizable llm pipelines instead of hand-wiring individual calls
• you want map-reduce style semantic operators with automatic parallelization
• you prefer python or yaml definitions for data transformation workflows

When to avoid

• your data is already structured and needs only deterministic transformations
• you cannot send your data to external llm providers due to privacy constraints
• you need real-time low-latency processing rather than batch pipelines
• you want a full visual etl platform without any coding

Facets

library · maturity active

etl llm-inference agent-framework nlp data-science workflow-automation cli large-language-models developer-tools python cli cross-platform llm-pipelines unstructured-data document-processing map-reduce semantic-data-processing data-pipelines docwrangler yaml-pipelines data-engineering natural-language-processing ai-agents automation

Member repositories

For agents

markdown · JSON · MCP: product_card(name="ucbepic/docetl")