# yahoo/TensorFlowOnSpark

TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

Repository: https://github.com/yahoo/TensorFlowOnSpark
Canonical: https://ross.abutalabs.com/products/tensorflowonspark
Language: Python
License: Apache-2.0
License Family: permissive
Topics: tensorflow, spark, yahoo, machine-learning, cluster, featured, python, scala
Last push: 2023-07-10T10:34:11+00:00

## Health v2 (maintenance only)
Score: 23/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 0, release rhythm 8, longevity 100
- inputs: {"age_days": 3512, "days_push": 1150, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 3845, forks 937 (observed 2026-08-28T04:08:24.061361+00:00)

## What it is
TensorFlowOnSpark is a Python library that lets existing TensorFlow programs run distributed training and inference on Apache Spark and Hadoop clusters with minimal code changes. It manages launching TensorFlow workers on Spark executors, data ingestion from HDFS or Spark RDDs, and cluster shutdown.

## Use cases
- run distributed TensorFlow training on a Spark cluster
- train deep learning models on data stored in HDFS
- migrate existing TensorFlow code to a shared Hadoop grid
- perform distributed inference across GPU and CPU servers
- integrate TensorFlow pipelines with Spark data processing
- scale deep learning workloads across a cluster of machines

## When to choose
- you already run Spark/Hadoop clusters and want TensorFlow training on them
- you want to migrate existing TensorFlow programs to distributed execution with minimal changes
- your training data lives in HDFS or Spark RDDs
- you need GPU or CPU distributed training on cloud or on-premise clusters

## When to avoid
- you use modern orchestration like Kubernetes-native TensorFlow or PyTorch distributed
- your workloads are small enough for a single machine
- you need a project with frequent updates and active development
- you use frameworks other than TensorFlow

## Facets
- artifact type: library
- maturity: maintenance
- function: machine-learning, deep-learning, llm-training
- domain: machine-learning, deep-learning, big-data, microservices
- platform: python, jvm, cloud
- tags: tensorflow, apache-spark, hadoop, distributed-training, gpu, hdfs, linux, docker

## Member repositories
- yahoo/TensorFlowOnSpark (main) score 23

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:08:24.061361+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T18:26:02.045102+00:00, confidence not recorded.
  - readme: https://github.com/yahoo/TensorFlowOnSpark (fetched 2026-08-28T04:08:24.061361+00:00, sha 1109a31cbb5f)
  - registry_pypi: https://pypi.org/pypi/tensorflowonspark/json (fetched 2026-08-29T09:20:47.825629+00:00, sha 6cd12721b9c2)
- Data as of 2026-08-30T08:39:29.467469+00:00.