apache/datafusion-ballista
Apache DataFusion Ballista Distributed Query Engine observed · 2026-08-28
Health v2 · maintenance only
98/100
- Activity 99
- Release rhythm 96
- Longevity 100
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-09-03. Adoption (stars, forks) is never an input.
- gap_med: 0.0
- age_days: 1567
- days_rel: 24
- days_push: 7
- n_releases_24m: 13
Adoption not part of the score
2115 stars · 312 forks observed · 2026-08-28
What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded
Apache DataFusion Ballista is a distributed query execution engine built on Apache DataFusion that parallelizes SQL and DataFrame workloads across multiple nodes. It offers Rust, Python, SQL, and CLI clients with a scheduler/executor architecture and blocking shuffle similar to Apache Spark.
Use cases
- run SQL queries across a distributed cluster
- scale DataFusion DataFrame workloads to multiple nodes
- execute large batch ETL jobs in parallel
- query Parquet and CSV files distributedly from Python
- replace Spark with a Rust-native query engine
- run distributed OLAP analytics on Arrow data
When to choose
- you already use DataFusion and need distributed execution with minimal code changes
- you run large, long-running batch or ETL jobs, especially on preemptible nodes
- you want a lightweight Rust-based alternative to Apache Spark
- you need Python bindings similar to PySpark for distributed queries
When to avoid
- your queries are interactive, short, and produce small intermediate results where a pipelined engine is faster
- you need full Apache Spark API compatibility rather than Spark-compatible functions
- your workload fits comfortably in a single process with DataFusion alone
- you need streaming shuffles between query stages
Facets
framework · maturity active
database etl streaming big-data microservices databases rust python cross-platform cli distributed-query-engine apache-arrow datafusion olap sql dataframe spark-alternative data-engineering
10 sources
- readme: https://github.com/apache/datafusion-ballista · fetched 2026-08-28 · d126042f90e5
- homepage: https://datafusion.apache.org/ballista · fetched 2026-08-29 · 8d4dd28c757b
- site_page: https://datafusion.apache.org/ballista/user-guide/deployment/cargo-install.html · fetched 2026-08-29 · d3ad84df69f4
- site_page: https://datafusion.apache.org/ballista/user-guide/python/quickstart.html · fetched 2026-08-29 · 045601be0181
- site_page: https://datafusion.apache.org/ballista/user-guide/faq.html · fetched 2026-08-29 · 1fc0b50b199e
- site_page: https://datafusion.apache.org/ballista/changelog/index.html · fetched 2026-08-29 · b7d335764d97
- site_page: https://datafusion.apache.org/ballista/changelog/54.1.0.html · fetched 2026-08-29 · 67b964a2eacc
- site_page: https://datafusion.apache.org/ballista/changelog/54.0.0.html · fetched 2026-08-29 · 3ac4f5bb7e6e
- site_page: https://datafusion.apache.org/ballista/changelog/53.0.0.html · fetched 2026-08-29 · e3c79bb6c5a0
- site_page: https://datafusion.apache.org/ballista/changelog/52.0.0.html · fetched 2026-08-29 · 2051e77995c0
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| apache/datafusion-ballista | main | 98 |
For agents
markdown · JSON · MCP: product_card(name="apache/datafusion-ballista")
Data as of 2026-08-30T08:39:29.467469+00:00 · Report a problem