chrislusf/gleam

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

3562 stars · 292 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Gleam is a fast, efficient distributed map/reduce execution system written in pure Go, defining computations as DAG flows that can run standalone or across a cluster. It supports in-memory or on-disk data flow, with user computations written in Go, Unix pipe tools, or any streaming programs.

Use cases

• run distributed map/reduce jobs in Go
• build ETL data pipelines with DAG execution
• process large datasets in memory or on disk
• execute data flows across a cluster of machines
• replace JVM-based big data frameworks with a lightweight Go alternative
• chain Unix command-line tools into distributed pipelines

When to choose

• you want a lightweight, memory-efficient distributed data processing system in pure Go
• you need DAG-based map/reduce execution that scales from a single machine to a cluster
• you want to mix Go code with Unix streaming tools in one pipeline

When to avoid

• you need a mature ecosystem with SQL, rich connectors, and large community support like Spark or Flink
• your workloads are small enough that a simple script suffices
• you need non-Go language SDKs beyond streaming programs

Facets

library · maturity active

etl streaming concurrency workflow-automation microservices big-data developer-tools windows go cross-platform map-reduce dag-execution distributed-computing data-pipeline unix-pipes data-engineering linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="chrislusf/gleam")