# databricks/koalas

Koalas: pandas API on Apache Spark

Repository: https://github.com/databricks/koalas
Canonical: https://ross.abutalabs.com/products/koalas
Language: Python
License: Apache-2.0
License Family: permissive
Topics: spark, pandas, pydata, dataframe, mlflow, big-data, data-science
Last push: 2024-03-20T15:33:34+00:00

## Health v2 (maintenance only)
Score: 23/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 0, release rhythm 8, longevity 100
- inputs: {"age_days": 2799, "days_push": 896, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 3372, forks 370 (observed 2026-08-28T04:07:58.421310+00:00)

## What it is
Koalas implements the pandas DataFrame API on top of Apache Spark, letting data scientists use familiar pandas code on distributed big data. It is now deprecated and merged into PySpark as of Apache Spark 3.2.

## Use cases
- run pandas code on large distributed datasets with spark
- scale existing pandas dataframes to big data
- use one codebase for pandas and spark
- migrate pandas workflows to spark without rewriting
- do data science on big data with familiar api

## When to choose
- you are on Spark 3.1 or below and want the pandas API on Spark
- you need a single codebase that works with both pandas and Spark
- your team knows pandas but not the Spark DataFrame API

## When to avoid
- you use Apache Spark 3.2 or above - use PySpark's built-in pandas API instead
- you are starting a new project - the project is deprecated and in maintenance mode
- your datasets fit comfortably in memory on a single machine - plain pandas is simpler

## Facets
- artifact type: library
- maturity: maintenance
- function: data-science, etl, streaming
- domain: big-data, data-science
- platform: python, cross-platform
- tags: pandas, apache-spark, dataframe, distributed-computing, deprecated, pyspark, data-engineering

## Member repositories
- databricks/koalas (main) score 23

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:07:58.421310+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T18:39:56.667068+00:00, confidence not recorded.
  - readme: https://github.com/databricks/koalas (fetched 2026-08-28T04:07:58.421310+00:00, sha 6eeccf23ddc5)
  - registry_pypi: https://pypi.org/pypi/koalas/json (fetched 2026-08-29T09:33:40.630248+00:00, sha b84fe66e0f39)
- Data as of 2026-08-30T08:39:29.467469+00:00.