treasure-data/digdag

Health v2 · maintenance only

Adoption not part of the score

1333 stars · 230 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Digdag is an open-source workload automation system for building, running, scheduling, and monitoring task pipelines as directed acyclic graphs (DAGs). It is written in Java, offers a CLI, server mode with a REST API and admin UI, and native Python and Ruby scripting support.

Use cases

• replace cron jobs with dependency-aware scheduled workflows
• orchestrate data engineering ETL pipelines across clouds
• coordinate machine learning training pipelines
• automate IT operations tasks with retries and error handling
• run workflows as code with versioning and rollbacks
• schedule and monitor batch analytics jobs

When to choose

• you need DAG-based dependency resolution instead of timing cron jobs
• you want a self-hosted scheduler with a web UI and REST API
• your pipelines mix Python, Ruby, shell, and cloud services like S3, Redshift, or BigQuery
• you need workflow-as-code with revisions, backfills, and resumable failures

When to avoid

• you want a large active ecosystem or frequent new features
• you need Kubernetes-native orchestration like Argo Workflows
• you prefer defining workflows in Python-only frameworks like Airflow or Prefect

Facets

application · maturity maintenance

workflow-automation scheduling etl cli web-framework self-hosted jvm cross-platform self-hosted workflow-engine dag job-scheduler data-pipelines cron-replacement workflow-as-code automation data-engineering devops docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="treasure-data/digdag")