# apache/datafusion-ballista

Apache DataFusion Ballista Distributed Query Engine

Repository: https://github.com/apache/datafusion-ballista
Canonical: https://ross.abutalabs.com/products/datafusion-ballista
Homepage: https://datafusion.apache.org/ballista
Language: Rust
License: Apache-2.0
License Family: permissive
Topics: arrow, big-data, dataframe, olap, python, query-engine, rust, sql, distributed
Last push: 2026-08-26T11:21:55+00:00

## Health v2 (maintenance only)
Score: 98/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 96, longevity 100
- inputs: {"age_days": 1567, "days_push": 7, "days_rel": 24, "gap_med": 0.0, "n_releases_24m": 13}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2115, forks 312 (observed 2026-08-28T04:06:15.831491+00:00)

## What it is
Apache DataFusion Ballista is a distributed query execution engine built on Apache DataFusion that parallelizes SQL and DataFrame workloads across multiple nodes. It offers Rust, Python, SQL, and CLI clients with a scheduler/executor architecture and blocking shuffle similar to Apache Spark.

## Use cases
- run SQL queries across a distributed cluster
- scale DataFusion DataFrame workloads to multiple nodes
- execute large batch ETL jobs in parallel
- query Parquet and CSV files distributedly from Python
- replace Spark with a Rust-native query engine
- run distributed OLAP analytics on Arrow data

## When to choose
- you already use DataFusion and need distributed execution with minimal code changes
- you run large, long-running batch or ETL jobs, especially on preemptible nodes
- you want a lightweight Rust-based alternative to Apache Spark
- you need Python bindings similar to PySpark for distributed queries

## When to avoid
- your queries are interactive, short, and produce small intermediate results where a pipelined engine is faster
- you need full Apache Spark API compatibility rather than Spark-compatible functions
- your workload fits comfortably in a single process with DataFusion alone
- you need streaming shuffles between query stages

## Facets
- artifact type: framework
- maturity: active
- function: database, etl, streaming
- domain: big-data, microservices, databases
- platform: rust, python, cross-platform, cli
- tags: distributed-query-engine, apache-arrow, datafusion, olap, sql, dataframe, spark-alternative, data-engineering

## Member repositories
- apache/datafusion-ballista (main) score 98

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:06:15.831491+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T02:53:21.550888+00:00, confidence not recorded.
  - readme: https://github.com/apache/datafusion-ballista (fetched 2026-08-28T04:06:15.831491+00:00, sha d126042f90e5)
  - homepage: https://datafusion.apache.org/ballista (fetched 2026-08-29T10:33:27.425745+00:00, sha 8d4dd28c757b)
  - site_page: https://datafusion.apache.org/ballista/user-guide/deployment/cargo-install.html (fetched 2026-08-29T10:33:27.435597+00:00, sha d3ad84df69f4)
  - site_page: https://datafusion.apache.org/ballista/user-guide/python/quickstart.html (fetched 2026-08-29T10:33:27.438466+00:00, sha 045601be0181)
  - site_page: https://datafusion.apache.org/ballista/user-guide/faq.html (fetched 2026-08-29T10:33:27.440955+00:00, sha 1fc0b50b199e)
  - site_page: https://datafusion.apache.org/ballista/changelog/index.html (fetched 2026-08-29T10:33:27.442993+00:00, sha b7d335764d97)
  - site_page: https://datafusion.apache.org/ballista/changelog/54.1.0.html (fetched 2026-08-29T10:33:27.444788+00:00, sha 67b964a2eacc)
  - site_page: https://datafusion.apache.org/ballista/changelog/54.0.0.html (fetched 2026-08-29T10:33:27.446664+00:00, sha 3ac4f5bb7e6e)
  - site_page: https://datafusion.apache.org/ballista/changelog/53.0.0.html (fetched 2026-08-29T10:33:27.449449+00:00, sha e3c79bb6c5a0)
  - site_page: https://datafusion.apache.org/ballista/changelog/52.0.0.html (fetched 2026-08-29T10:33:27.452218+00:00, sha 2051e77995c0)
- Data as of 2026-08-30T08:39:29.467469+00:00.