bytewax/bytewax

Health v2 · maintenance only

Adoption not part of the score

2047 stars · 111 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Bytewax is a Python-first framework with a Rust-based distributed engine for stateful event and stream processing, inspired by Apache Flink, Spark, and Kafka Streams. It lets developers compose dataflow pipelines with operators like map, filter, join, and fold_window, scaling from local development to multi-node Kubernetes deployments.

Use cases

• process kafka streams in python
• build stateful event-driven pipelines
• run online machine learning on streaming data
• aggregate and window real-time events
• join and enrich live data streams
• deploy distributed stream processing on kubernetes

When to choose

• you want Flink-like stream processing but in Python
• you need stateful processing with automatic state recovery
• you want to use Python libraries directly on streaming data
• you need to scale pipelines across multiple workers or nodes

When to avoid

• you need a vendor-backed project with strong commercial support (the company wound down in 2025)
• you need JVM-native Flink or Kafka Streams features
• you only need simple batch ETL rather than streaming
• you require guaranteed long-term maintenance from a large team

Facets

framework · maturity maintenance

streaming etl machine-learning data-science big-data machine-learning python rust windows stream-processing dataflow event-processing kafka stateful-processing online-machine-learning data-engineering real-time docker kubernetes linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="bytewax/bytewax")