# vaexio/vaex

Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀

Repository: https://github.com/vaexio/vaex
Canonical: https://ross.abutalabs.com/products/vaex
Homepage: https://vaex.io
Language: Python
License: MIT
License Family: permissive
Topics: dataframe, python, bigdata, tabular-data, visualization, memory-mapped-file, hdf5, machine-learning, machinelearning, data-science, pyarrow
Last push: 2026-04-01T03:56:14+00:00

## Health v2 (maintenance only)
Score: 57/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 75, release rhythm 8, longevity 100
- inputs: {"age_days": 4358, "days_push": 154, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 8507, forks 601 (observed 2026-08-28T04:10:23.192213+00:00)

## What it is
Vaex is a high-performance Python DataFrame library for lazy, out-of-core processing of large tabular datasets, using memory mapping and zero-copy operations to process over a billion rows per second. It includes visualization, statistics on N-dimensional grids, fast groupby/join operations, and a machine-learning subpackage, with a Pandas-like API.

## Use cases
- analyze a 100GB CSV file that doesn't fit in RAM
- compute statistics on billions of rows quickly
- visualize and explore large tabular datasets interactively in Jupyter
- convert huge CSV files to HDF5 or Arrow format
- do fast groupby aggregations on big data
- build machine learning pipelines on large datasets on a laptop
- join two large tables without copying data into memory

## When to choose
- your tabular data exceeds available memory and you want to avoid setting up a cluster
- you need fast statistics, groupby, or joins on billions of rows on a single machine
- you want interactive visualization of large datasets in Jupyter
- you want a Pandas-like API for out-of-core data processing

## When to avoid
- your datasets fit comfortably in memory and Pandas or Polars suffice
- you need distributed computing across a cluster (use Spark or Dask)
- you need heavy ecosystem support or very active development
- you work with unstructured data like text or images rather than tabular data

## Facets
- artifact type: library
- maturity: maintenance
- function: data-science, data-visualization, machine-learning, etl
- domain: data-science, big-data, data-visualization, machine-learning
- platform: python, cross-platform
- tags: dataframe, out-of-core, memory-mapping, hdf5, apache-arrow, big-data, jupyter

## Member repositories
- vaexio/vaex (main) score 57

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:10:23.192213+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T17:26:42.210105+00:00, confidence not recorded.
  - readme: https://github.com/vaexio/vaex (fetched 2026-08-28T04:10:23.192213+00:00, sha 7245acfa0d6e)
  - homepage: https://vaex.io (fetched 2026-08-29T08:26:56.261404+00:00, sha 52f6286d924e)
  - site_page: https://vaex.io/docs/index.html (fetched 2026-08-29T08:26:56.271080+00:00, sha a8b14104785a)
  - site_page: https://vaex.io/docs/installing.html (fetched 2026-08-29T08:26:56.273479+00:00, sha 24970d7e3dc7)
  - site_page: https://vaex.io/docs/tutorials.html (fetched 2026-08-29T08:26:56.275192+00:00, sha 13143ce1860d)
  - site_page: https://vaex.io/docs/guides.html (fetched 2026-08-29T08:26:56.276785+00:00, sha 7a6145d763e1)
  - site_page: https://vaex.io/docs/faq.html (fetched 2026-08-29T08:26:56.280218+00:00, sha 6740eac4fe2c)
  - site_page: https://vaex.io/blog/8-incredibly-powerful-Vaex-features-you-might-have-not-known-about (fetched 2026-08-29T08:26:56.281898+00:00, sha 23ebdfb9e50d)
- Data as of 2026-08-30T08:39:29.467469+00:00.