# cgarciae/pypeln

Concurrent data pipelines in Python   >>>

Repository: https://github.com/cgarciae/pypeln
Canonical: https://ross.abutalabs.com/products/pypeln
Homepage: https://cgarciae.github.io/pypeln
Language: Python
License: MIT
License Family: permissive
Last push: 2023-07-20T16:18:20+00:00

## Health v2 (maintenance only)
Score: 23/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 0, release rhythm 8, longevity 100
- inputs: {"age_days": 2923, "days_push": 1140, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1596, forks 94 (observed 2026-08-28T04:05:09.124564+00:00)

## What it is
Pypeln is a Python library for building concurrent, multi-stage data pipelines using processes, threads, or asyncio tasks through a single functional API. It targets medium-sized data workloads where heavyweight frameworks like Spark or Dask are overkill.

## Use cases
- parallelize map/filter over large iterables
- build multi-stage concurrent data pipelines in Python
- process medium data with bounded memory using maxsize
- run IO-bound tasks concurrently with asyncio tasks
- speed up CPU-bound transformations with multiprocessing
- control worker counts per pipeline stage

## When to choose
- you need concurrent pipelines for medium data without Spark/Dask overhead
- you want one API spanning processes, threads, and asyncio
- you need fine-grained control over workers and queue sizes per stage

## When to avoid
- you need distributed computing across a cluster
- your data is small enough that plain Python is fast enough
- you need a maintained project with active development

## Facets
- artifact type: library
- maturity: maintenance
- function: concurrency, streaming, etl, developer-tools
- domain: developer-tools
- platform: python, cross-platform
- tags: data-pipelines, parallel-processing, asyncio, multiprocessing, functional-api, data-engineering, automation, algorithms

## Member repositories
- cgarciae/pypeln (main) score 23

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:09.124564+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T03:54:02.519842+00:00, confidence not recorded.
  - readme: https://github.com/cgarciae/pypeln (fetched 2026-08-28T04:05:09.124564+00:00, sha 2d423296f06e)
  - homepage: https://cgarciae.github.io/pypeln (fetched 2026-08-29T11:24:44.346106+00:00, sha 21ccead20fac)
  - registry_pypi: https://pypi.org/pypi/pypeln/json (fetched 2026-08-29T11:24:44.348781+00:00, sha 9ce51cc4ec38)
- Data as of 2026-08-30T08:39:29.467469+00:00.