twitter/summingbird

Health v2 · maintenance only

Flags: archived

Adoption not part of the score

2123 stars · 256 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Summingbird is a Scala library for writing MapReduce aggregation programs that look like native collection transformations and run on distributed platforms such as Storm (realtime) and Scalding (batch), including a hybrid batch/realtime mode. It is officially retired by Twitter.

Use cases

• write streaming mapreduce aggregations in scala
• run the same aggregation logic in batch and realtime
• build fault-tolerant hybrid batch/streaming pipelines on storm and scalding
• word counting over large data streams
• sum key-value events into a distributed store

When to choose

• you already run Scalding or Storm on the JVM and want unified batch/realtime aggregation code
• you need hybrid batch/realtime fault-tolerance semantics and can maintain the library yourself

When to avoid

• starting a new project - the project is officially retired and unmaintained
• you use modern stream processors like Flink, Kafka Streams, or Spark Streaming
• you are not on the JVM/Scala ecosystem

Facets

library · maturity abandoned

streaming etl data-science big-data microservices jvm mapreduce storm scalding scala batch-realtime-hybrid retired data-engineering

Member repositories

For agents

markdown · JSON · MCP: product_card(name="twitter/summingbird")