apache/datafusion
Apache DataFusion SQL Query Engine observed · 2026-08-28
Health v2 · maintenance only
77/100
- Activity 99
- Release rhythm 35
- Longevity 100
Flags: no_releases
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-09-03. Adoption (stars, forks) is never an input.
- gap_med: n/a
- age_days: 1964
- days_rel: n/a
- days_push: 7
- n_releases_24m: 0
Adoption not part of the score
9202 stars · 2343 forks observed · 2026-08-28
What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded
Apache DataFusion is an extensible query engine written in Rust that uses Apache Arrow as its in-memory columnar format. It provides SQL and DataFrame APIs with a full query planner and a vectorized, multi-threaded, streaming execution engine designed to be embedded into custom database and analytics systems, with Python bindings and a datafusion-cli binary.
Use cases
- run SQL queries directly on parquet files
- embed a SQL query engine in my Rust application
- build a custom database or analytics product without writing an execution engine
- query CSV and JSON files with SQL from the command line
- use a DataFrame API for analytics in Rust or Python
- fast in-process OLAP analytics on columnar data
When to choose
- You are a developer building a database, analytics platform, or query system and want reusable SQL parsing, planning, and vectorized execution rather than starting from scratch
- You need fast single-process queries over Parquet, CSV, JSON, or Avro using the Apache Arrow memory model
- You need deep customization points such as custom data sources, operators, functions, window/aggregate functions, or alternative query languages
- You prefer an Apache Software Foundation-governed, actively developed engine with an active community and Python bindings
When to avoid
- You want an end-user ready serverless analytical database to use directly, where DuckDB is a better fit
- You want a batteries-included DataFrame library aimed at data scientists like Polars
- You need distributed multi-node execution out of the box, which is handled by the separate Ballista project
- You need a transactional OLTP database rather than an analytical query engine
Facets
library · maturity active
database parser cli data-science databases analytics big-data data-science rust python cli cross-platform sql query-engine dataframe apache-arrow columnar olap parquet in-process embeddable streaming-execution query-optimizer
6 sources
- readme: https://github.com/apache/datafusion · fetched 2026-08-28 · 06a614fee7fc
- homepage: https://datafusion.apache.org/ · fetched 2026-08-29 · 25e945e4ad2d
- site_page: https://datafusion.apache.org/user-guide/features.html · fetched 2026-08-29 · 21693b9b77f0
- site_page: https://datafusion.apache.org/user-guide/cli/installation.html · fetched 2026-08-29 · dd28e4dec4ab
- site_page: https://datafusion.apache.org/user-guide/faq.html · fetched 2026-08-29 · aa71679b0111
- registry_crates: https://crates.io/api/v1/crates/datafusion · fetched 2026-08-29 · d9825b58cedd
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| apache/datafusion | main | 77 |
For agents
markdown · JSON · MCP: product_card(name="apache/datafusion")
Data as of 2026-08-30T08:39:29.467469+00:00 · Report a problem