# kubeflow/spark-operator

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.

Repository: https://github.com/kubeflow/spark-operator
Canonical: https://ross.abutalabs.com/products/spark-operator
Homepage: https://spark.kubeflow.org/en/latest/
Language: Python
License: Apache-2.0
License Family: permissive
Topics: kubernetes, kubernetes-operator, apache-spark, kubernetes-crd, kubernetes-controller, spark, google-cloud-dataproc
Last push: 2026-08-26T02:42:08+00:00

## Health v2 (maintenance only)
Score: 94/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 99, release rhythm 83, longevity 100
- inputs: {"age_days": 3164, "days_push": 7, "days_rel": 33, "gap_med": 61, "n_releases_24m": 12}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 3149, forks 1519 (observed 2026-08-28T04:07:46.054721+00:00)

## What it is
A Kubernetes operator that manages the full lifecycle of Apache Spark applications using custom resources (SparkApplication CRDs). It lets you declare, run, and monitor Spark jobs on Kubernetes just like any other native workload.

## Use cases
- run spark jobs on kubernetes
- manage spark application lifecycle with a kubernetes operator
- deploy sparkapplication crd via helm
- schedule batch spark workloads in a k8s cluster
- monitor spark job status with kubectl
- migrate spark workloads from dataproc to kubernetes

## When to choose
- you already run workloads on Kubernetes and want Spark jobs managed declaratively with CRDs
- you need automated retries, status reporting, and lifecycle management for Spark applications
- you want to install via Helm or Kustomize into an existing cluster

## When to avoid
- you run Spark on dedicated clusters or managed services like Databricks or EMR instead of Kubernetes
- you need a lightweight way to submit one-off Spark jobs without a controller installed
- your cluster cannot run the operator's RBAC and CRDs

## Facets
- artifact type: infra-config
- maturity: active
- function: container-orchestration, deployment, workflow-automation, scheduling
- domain: big-data, cloud-computing
- platform: cloud, go, python
- tags: kubernetes-operator, apache-spark, crd, helm-chart, spark-applications, dataproc, data-engineering, containers, devops, kubernetes, docker

## Member repositories
- kubeflow/spark-operator (main) score 94

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:07:46.054721+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T07:25:55.443954+00:00, confidence not recorded.
  - readme: https://github.com/kubeflow/spark-operator (fetched 2026-08-28T04:07:46.054721+00:00, sha ad66127db9a8)
- Data as of 2026-08-30T08:39:29.467469+00:00.