# hi-primus/optimus

:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

Repository: https://github.com/hi-primus/optimus
Canonical: https://ross.abutalabs.com/products/optimus
Homepage: https://hi-optimus.com
Language: Python
License: Apache-2.0
License Family: permissive
Topics: spark, pyspark, data-wrangling, bigdata, big-data-cleaning, data-science, data-cleansing, data-cleaner, data-transformation, machine-learning, data-profiling, data-extraction, data-exploration, data-analysis, data-preparation, cudf, dask, dask-cudf, data-cleaning
Last push: 2024-12-02T14:09:25+00:00

## Health v2 (maintenance only)
Score: 23/100 (v2, computed 2026-09-03T02:39:23.370411+00:00)
- activity 0, release rhythm 8, longevity 100
- inputs: {"age_days": 3339, "days_push": 639, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1536, forks 231 (observed 2026-08-28T04:05:00.008181+00:00)

## What it is
Optimus is a Python library for agile data preparation that provides a unified API over pandas, Dask, cuDF, Dask-cuDF, Vaex, and PySpark. It offers 100+ functions for string, date, URL, and email processing, plus data quality exploration, plotting, and ML model creation that scales from a laptop to a GPU cluster.

## Use cases
- clean and transform large datasets with a single API
- profile and fix data quality issues
- process data on a laptop and scale to a Spark or GPU cluster without rewriting code
- explore and plot data of any size
- prepare data for machine learning pipelines

## When to choose
- you want one data-processing API that works across pandas, Dask, cuDF, Vaex, and Spark backends
- you need out-of-the-box data cleaning and quality-fixing functions
- you want to move the same workflow from a laptop to a GPU or Spark cluster

## When to avoid
- you only need plain pandas and prefer its native API
- you need fine-grained control over backend-specific optimizations
- your team already has an established ETL stack and doesn't want an opinionated wrapper

## Facets
- artifact type: library
- maturity: active
- function: etl, data-science, machine-learning, data-visualization
- domain: data-science, big-data, machine-learning
- platform: python
- tags: data-cleaning, data-wrangling, data-profiling, pandas, dask, cudf, pyspark, vaex, data-preparation, data-engineering, gpu, docker

## Member repositories
- hi-primus/optimus (main) score 23

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:00.008181+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T04:31:04.862952+00:00, confidence not recorded.
  - readme: https://github.com/hi-primus/optimus (fetched 2026-08-28T04:05:00.008181+00:00, sha 9963fe51c54a)
  - homepage: https://hi-optimus.com (fetched 2026-08-29T11:32:57.752376+00:00, sha 238cb44dbfa1)
- Data as of 2026-08-30T08:39:29.467469+00:00.