hi-primus/optimus

Health v2 · maintenance only

Adoption not part of the score

1536 stars · 231 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Optimus is a Python library for agile data preparation that provides a unified API over pandas, Dask, cuDF, Dask-cuDF, Vaex, and PySpark. It offers 100+ functions for string, date, URL, and email processing, plus data quality exploration, plotting, and ML model creation that scales from a laptop to a GPU cluster.

Use cases

• clean and transform large datasets with a single API
• profile and fix data quality issues
• process data on a laptop and scale to a Spark or GPU cluster without rewriting code
• explore and plot data of any size
• prepare data for machine learning pipelines

When to choose

• you want one data-processing API that works across pandas, Dask, cuDF, Vaex, and Spark backends
• you need out-of-the-box data cleaning and quality-fixing functions
• you want to move the same workflow from a laptop to a GPU or Spark cluster

When to avoid

• you only need plain pandas and prefer its native API
• you need fine-grained control over backend-specific optimizations
• your team already has an established ETL stack and doesn't want an opinionated wrapper

Facets

library · maturity active

etl data-science machine-learning data-visualization data-science big-data machine-learning python data-cleaning data-wrangling data-profiling pandas dask cudf pyspark vaex data-preparation data-engineering gpu docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="hi-primus/optimus")