# spotify/luigi

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

Repository: https://github.com/spotify/luigi
Canonical: https://ross.abutalabs.com/products/luigi
Language: Python
License: Apache-2.0
License Family: permissive
Topics: python, luigi, orchestration-framework, scheduling, hadoop
Last push: 2026-07-18T11:15:46+00:00

## Health v2 (maintenance only)
Score: 91/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 93, release rhythm 83, longevity 100
- inputs: {"age_days": 5095, "days_push": 46, "days_rel": 118, "gap_med": 22, "n_releases_24m": 8}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 18765, forks 2464 (observed 2026-08-28T04:11:27.113280+00:00)

## What it is
Luigi is a Python package for building complex pipelines of long-running batch jobs. It handles dependency resolution, workflow management, failure handling, and visualization, with built-in Hadoop, Spark, and Hive integrations.

## Use cases
- orchestrate batch data pipelines in python
- resolve task dependencies in etl workflows
- schedule hadoop and spark jobs
- build multi-step data processing pipelines
- visualize and monitor long-running batch workflows

## When to choose
- you need to chain many batch tasks with dependency resolution
- your pipelines include hadoop, spark, hive, or database jobs
- you want a mature, battle-tested python workflow library

## When to avoid
- you need real-time or streaming data processing
- you prefer a modern dag-based orchestrator with a rich web UI like Airflow or Prefect

## Facets
- artifact type: library
- maturity: stable
- function: workflow-automation, scheduling, etl
- domain: big-data
- platform: python, cli
- tags: batch-jobs, hadoop, spark, data-pipelines, task-dependency-resolution, data-engineering, automation

## Member repositories
- spotify/luigi (main) score 91

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:11:27.113280+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T17:01:18.151815+00:00, confidence not recorded.
  - readme: https://github.com/spotify/luigi (fetched 2026-08-28T04:11:27.113280+00:00, sha a9ad3d5e9cd7)
  - registry_pypi: https://pypi.org/pypi/luigi/json (fetched 2026-08-29T07:59:14.778916+00:00, sha 3092fc4bf226)
- Data as of 2026-08-30T08:39:29.467469+00:00.