vortex-data/vortex

Health v2 · maintenance only

Adoption not part of the score

3158 stars · 211 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Vortex is an extensible, high-performance columnar file format and toolkit for compressed data, positioned as a faster alternative to Apache Parquet. It provides zero-copy Apache Arrow compatibility, pluggable encodings, and integrations with query engines like DataFusion, DuckDB, Spark, and Trino.

Use cases

• compress columnar data faster than parquet
• speed up random access reads on object storage
• convert parquet files to a faster columnar format
• query compressed data with duckdb or datafusion
• build a data system backed by object storage
• read and write columnar files from python or rust
• accelerate analytics scans over wide tables

When to choose

• you need faster scans or random access than Apache Parquet
• you work in the Rust, Python, Java, or C++ data ecosystem and want zero-copy Arrow interop
• you want pluggable encodings and a stable, extensible file format
• you integrate with DataFusion, DuckDB, Spark, Trino, or Ray

When to avoid

• you need maximum ecosystem maturity and universal tooling support that Parquet enjoys
• your pipeline requires Iceberg integration today (still coming soon)
• you depend on stable library APIs, which may still change between versions

Facets

library · maturity active

serialization compression database etl databases big-data analytics developer-tools rust python cross-platform cli columnar-format apache-arrow parquet-alternative file-format data-lake query-engine-integration zero-copy fsst fastlanes data-engineering

Member repositories

For agents

markdown · JSON · MCP: product_card(name="vortex-data/vortex")