# apache/beam

Apache Beam is a unified programming model for Batch and Streaming data processing.

Repository: https://github.com/apache/beam
Canonical: https://ross.abutalabs.com/products/beam
Homepage: https://beam.apache.org/
Language: Java
License: Apache-2.0
License Family: permissive
Topics: python, java, big-data, beam, batch, golang, sql, streaming
Last push: 2026-08-26T22:02:30+00:00

## Health v2 (maintenance only)
Score: 93/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 80, longevity 100
- inputs: {"age_days": 3865, "days_push": 7, "days_rel": 56, "gap_med": 40, "n_releases_24m": 16}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 8650, forks 4630 (observed 2026-08-28T04:10:24.018712+00:00)

## What it is
Apache Beam is an open-source unified programming model and SDK set (Java, Python, Go, SQL, TypeScript) for defining batch and streaming data-parallel processing pipelines. Pipelines are portable across execution runners including Apache Flink, Apache Spark, Google Cloud Dataflow, Samza, and Hazelcast Jet.

## Use cases
- build etl pipelines that run on multiple engines
- process streaming data in near real time at scale
- run batch and streaming with one codebase
- run ml model inference over large datasets
- migrate pipelines between spark flink and dataflow
- process trillions of events daily in production

## When to choose
- you need one pipeline codebase for both batch and streaming
- you want portability across Flink, Spark, Dataflow, and other runners
- you need large-scale distributed data processing with strong abstractions
- you want to run ML inference as part of big data pipelines

## When to avoid
- you have small datasets that fit on one machine
- you want a simple scheduler rather than a full programming model
- your team cannot invest in learning the Beam model
- you are locked into a single engine's native APIs and don't need portability

## Facets
- artifact type: framework
- maturity: stable
- function: etl, streaming, data-science, machine-learning
- domain: big-data, microservices, data-science, machine-learning
- platform: python, jvm, go, cross-platform, cloud
- tags: batch-processing, stream-processing, data-pipelines, apache-flink, apache-spark, google-cloud-dataflow, runinference, multi-language-pipelines, unified-programming-model, sql, data-engineering, docker

## Member repositories
- apache/beam (main) score 93

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:10:24.018712+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T17:25:40.603051+00:00, confidence not recorded.
  - readme: https://github.com/apache/beam (fetched 2026-08-28T04:10:24.018712+00:00, sha 945424fd2968)
  - homepage: https://beam.apache.org/ (fetched 2026-08-29T08:26:11.890597+00:00, sha f9ace367bd9a)
  - site_page: https://beam.apache.org/documentation (fetched 2026-08-29T08:26:11.893583+00:00, sha 7f345db5369d)
  - site_page: https://beam.apache.org/about (fetched 2026-08-29T08:26:11.895563+00:00, sha 7fa8d4f202a5)
  - site_page: https://beam.apache.org/documentation/sdks/java (fetched 2026-08-29T08:26:11.897412+00:00, sha 08948fc00723)
  - site_page: https://beam.apache.org/documentation/runners/capability-matrix (fetched 2026-08-29T08:26:11.899152+00:00, sha a29c6c9a3518)
  - site_page: https://beam.apache.org/documentation/io/connectors (fetched 2026-08-29T08:26:11.901094+00:00, sha de88e0eaf904)
  - site_page: https://beam.apache.org/documentation/ml/overview (fetched 2026-08-29T08:26:11.902976+00:00, sha 35dd6c6fcd01)
  - site_page: https://beam.apache.org/get-started/quickstart-java (fetched 2026-08-29T08:26:11.904693+00:00, sha 4d7d165d641a)
  - site_page: https://beam.apache.org/get-started/quickstart-py (fetched 2026-08-29T08:26:11.906511+00:00, sha e0c8e00fbdfc)
- Data as of 2026-08-30T08:39:29.467469+00:00.