apache/airflow

Health v2 · maintenance only

Adoption not part of the score

46613 stars · 17684 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Apache Airflow is an open-source platform for programmatically authoring, scheduling, and monitoring workflows as directed acyclic graphs (DAGs) defined in Python. It provides a web UI, scheduler, and a rich ecosystem of operators and hooks for integrating with cloud services and data tools.

Use cases

• schedule daily ETL pipelines that extract and load data into a warehouse
• orchestrate machine learning training and deployment workflows
• replace cron jobs with a monitored, dependency-aware scheduler
• build data integration pipelines across AWS, GCP, and Azure services
• trigger and monitor backfills of historical data processing
• coordinate multi-step batch jobs with retries and alerting

When to choose

• you need to orchestrate complex, dependency-driven batch workflows in Python
• you want a mature, battle-tested scheduler with a web UI and large community
• your pipelines integrate with many external systems via existing operators and hooks
• you need dynamic pipeline generation and Jinja templating

When to avoid

• you need low-latency, event-driven streaming rather than scheduled batch workflows
• your use case is simple cron-style scheduling without dependencies or monitoring needs
• you want fully managed orchestration without operating scheduler infrastructure yourself

Facets

framework · maturity stable

workflow-automation scheduling etl streaming monitoring developer-tools data-science machine-learning big-data python self-hosted cli cloud workflow-orchestration dag data-pipelines scheduler elt mlops data-integration airflow data-engineering automation devops docker kubernetes web-server

Member repositories

For agents

markdown · JSON · MCP: product_card(name="apache/airflow")