# arkflow-rs/arkflow

High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.

Repository: https://github.com/arkflow-rs/arkflow
Canonical: https://ross.abutalabs.com/products/arkflow
Homepage: https://arkflow-rs.com/
Language: Rust
License: Apache-2.0
License Family: permissive
Topics: flow, rust, rust-lang, stream, sql, datafusion, duckdb, kafka, mysql, postgresql, sqlite, tokio, tokio-rs, arkflow, redis, nats, websocket, ai, deep-learning, machine-learning
Last push: 2026-08-16T00:34:51+00:00

## Health v2 (maintenance only)
Score: 70/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 97, release rhythm 53, longevity 39
- inputs: {"age_days": 550, "days_push": 18, "days_rel": 318, "gap_med": 9, "n_releases_24m": 8}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1302, forks 46 (observed 2026-08-28T04:04:18.178470+00:00)

## What it is
ArkFlow is a high-performance stream processing engine written in Rust on top of Tokio, connecting configurable inputs (Kafka, MQTT, HTTP, files, SQL databases), processors (SQL via DataFusion, JSON, Protobuf, VRL, Python UDFs), and outputs into YAML-defined pipelines. It integrates AI capabilities such as loading machine learning models for streaming inference and anomaly detection, and offers an optional control plane for operating fleets of nodes.

## Use cases
- process kafka streams with sql queries in real time
- run machine learning inference on streaming data
- build an iot mqtt data pipeline
- transform and route  events between systems
- process debezium cdc change streams
- detect anomalies in real-time sensor data

## When to choose
- you need a lightweight, high-performance alternative to Flink or Kafka Streams for stateless pipelines
- you want SQL, VRL, or Python UDF processing over streaming data in a single YAML-configured binary
- you need streaming inference or anomaly detection with ML models embedded in the pipeline

## When to avoid
- you need full stateful stream processing with windowing, joins, and exactly-once state management today
- you require a mature, battle-tested engine with a large community and ecosystem
- your pipelines depend on connectors or processors not yet supported by its modular component set

## Facets
- artifact type: application
- maturity: active
- function: streaming, etl, machine-learning, serialization, message-queue
- domain: big-data, machine-learning, iot
- platform: windows, rust, self-hosted
- tags: stream-processing, datafusion, kafka, mqtt, cdc, vrl, arrow, tokio, real-time-pipelines, ai-inference, sql, data-engineering, real-time, linux, macos, docker

## Member repositories
- arkflow-rs/arkflow (main) score 70

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:04:18.178470+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T04:51:16.535911+00:00, confidence not recorded.
  - readme: https://github.com/arkflow-rs/arkflow (fetched 2026-08-28T04:04:18.178470+00:00, sha d0c2d7960d0d)
  - homepage: https://arkflow-rs.com/ (fetched 2026-08-29T12:09:28.177840+00:00, sha 5eb01aa1a684)
  - site_page: https://arkflow-rs.com/docs/intro (fetched 2026-08-29T12:09:28.180560+00:00, sha 824ab7f37b7a)
  - site_page: https://arkflow-rs.com/docs/next/start-here (fetched 2026-08-29T12:09:28.182644+00:00, sha ca379148a426)
  - site_page: https://arkflow-rs.com/docs/0.3.x/intro (fetched 2026-08-29T12:09:28.184223+00:00, sha 67265ccad1a4)
  - site_page: https://arkflow-rs.com/docs/0.2.x/intro (fetched 2026-08-29T12:09:28.185963+00:00, sha 087e15b2de86)
- Data as of 2026-08-30T08:39:29.467469+00:00.