# Pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

Repository: https://github.com/pathwaycom/pathway
Canonical: https://ross.abutalabs.com/products/pathway
Homepage: https://pathway.com
Language: Python
License: NOASSERTION
License Family: other
Topics: batch-processing, kafka, pathway, python, streaming, machine-learning-algorithms, real-time, data-analytics, data-pipelines, data-processing, dataflow, etl, etl-framework, iot-analytics, rust, stream-processing, time-series-analysis
Last push: 2026-08-26T05:38:32+00:00

## Health v2 (maintenance only)
Score: 97/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 99, release rhythm 95, longevity 98
- inputs: {"age_days": 1375, "days_push": 7, "days_rel": 32, "gap_med": 13.0, "n_releases_24m": 43}
- flags: no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 62383, forks 1679 (observed 2026-08-28T04:12:19.108825+00:00)

## What it is
Pathway is a Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG applications. It provides a unified Python API that handles both batch and streaming data in development and production, backed by a Rust engine.

## Use cases
- build real-time data pipelines from kafka streams
- process streaming data with python
- build a rag pipeline over live documents
- run llm pipelines on continuously updating data
- perform real-time analytics on iot sensor data
- unify batch and streaming etl in one framework
- index and query changing data for ai applications

## When to choose
- you need one codebase for both batch and streaming data
- you are building RAG or LLM pipelines over continuously updating sources
- you want a Python API with a high-performance Rust runtime
- you need real-time analytics on kafka or iot streams

## When to avoid
- you need a permissively licensed library for commercial redistribution (BSL license)
- you only need simple one-off batch transformations where pandas suffices
- you require a mature SQL-first streaming system like Flink

## Facets
- artifact type: framework
- maturity: active
- function: etl, streaming, rag, machine-learning, data-science, llm-inference
- domain: big-data, large-language-models, analytics
- platform: python, self-hosted, cross-platform
- tags: stream-processing, real-time-analytics, llm-pipelines, kafka, rust-engine, bsl-license, data-engineering, retrieval-augmented-generation, real-time, docker

## Member repositories
- pathwaycom/pathway (main) score 97
- pathwaycom/llm-app (examples) score 69

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:12:19.108825+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T16:17:28.013868+00:00, confidence not recorded.
  - readme: https://github.com/pathwaycom/pathway (fetched 2026-08-28T04:12:19.108825+00:00, sha 916027a95e48)
- Data as of 2026-08-30T08:39:29.467469+00:00.