# lensacom/sparkit-learn

PySpark + Scikit-learn = Sparkit-learn

Repository: https://github.com/lensacom/sparkit-learn
Canonical: https://ross.abutalabs.com/products/sparkit-learn
Language: Python
License: Apache-2.0
License Family: permissive
Topics: scikit-learn, apache-spark, machine-learning, distributed-computing, python
Last push: 2020-12-31T01:56:49+00:00

## Health v2 (maintenance only)
Score: 23/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 0, release rhythm 8, longevity 100
- inputs: {"age_days": 4340, "days_push": 2072, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1150, forks 254 (observed 2026-08-28T04:03:46.643349+00:00)

## What it is
Sparkit-learn provides scikit-learn's API and functionality on top of PySpark, operating on distributed RDDs of numpy arrays and sparse matrices. It lets users run familiar sklearn-style estimators and transformers across a Spark cluster.

## Use cases
- run scikit-learn models on a spark cluster
- distributed machine learning with pyspark
- scale sklearn transformers across partitions
- train models on large datasets that don't fit in memory
- use sklearn API on RDDs

## When to choose
- you already have a Spark cluster and want sklearn-style APIs
- your data lives in RDDs and you need distributed preprocessing and modeling

## When to avoid
- you need actively maintained software (last release 2020, supports old Python/Spark versions)
- you use Spark DataFrames rather than RDDs
- you want modern distributed ML like Spark MLlib or Dask-ML

## Facets
- artifact type: library
- maturity: maintenance
- function: machine-learning, etl
- domain: machine-learning, data-science, big-data
- platform: python, jvm, cross-platform
- tags: pyspark, scikit-learn, distributed-computing, rdd, apache-spark

## Member repositories
- lensacom/sparkit-learn (main) score 23

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:03:46.643349+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T06:33:40.834150+00:00, confidence not recorded.
  - readme: https://github.com/lensacom/sparkit-learn (fetched 2026-08-28T04:03:46.643349+00:00, sha a28f3050f14f)
  - registry_pypi: https://pypi.org/pypi/sparkit-learn/json (fetched 2026-08-29T12:38:32.184052+00:00, sha 2c559662630c)
- Data as of 2026-08-30T08:39:29.467469+00:00.