apache/auron

Health v2 · maintenance only

Adoption not part of the score

1795 stars · 241 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Apache Auron (Incubating) is a native vectorized query accelerator for distributed big data engines like Apache Spark, built on Apache DataFusion and Apache Arrow. It translates Spark's optimized physical plans into Rust-based native execution plans, eliminating JVM overhead and delivering roughly 2x faster TPC-DS performance with lower resource usage.

Use cases

• speed up Apache Spark SQL queries with native vectorized execution
• reduce cluster resource costs for large-scale Spark workloads
• run TPC-DS style analytical queries faster on Spark
• avoid JVM overhead in distributed SQL execution
• integrate DataFusion-based native operators into a Spark pipeline
• support data lake formats like Hudi and Paimon with accelerated queries

When to choose

• you run large-scale Spark SQL workloads and need better performance or lower resource usage
• you want production-hardened native execution with fine-grained failback to Spark
• you use HDFS, S3, Hudi, Paimon, or Celeborn in your Spark stack
• you need predictable performance without rewriting existing Spark jobs

When to avoid

• your workloads are small or not SQL/Spark-centric
• you cannot add a custom shuffle manager and Spark extensions to your cluster
• you need a fully graduated (non-incubating) Apache project with long-term guarantees
• you rely on operators or UDFs not yet supported by Auron's native engine

Facets

library · maturity active

plugin-system big-data analytics microservices performance jvm rust cloud apache-spark datafusion apache-arrow native-execution simd tpc-ds query-optimizer shuffle sql columnar query-execution vectorized-execution query-acceleration sql-engine data-engineering linux docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="apache/auron")