# Yelp/mrjob

Run MapReduce jobs on Hadoop or Amazon Web Services

Repository: https://github.com/Yelp/mrjob
Canonical: https://ross.abutalabs.com/products/mrjob
Homepage: http://packages.python.org/mrjob/
Language: Python
License: NOASSERTION
License Family: other
Last push: 2026-04-02T17:05:58+00:00

## Health v2 (maintenance only)
Score: 66/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 75, release rhythm 35, longevity 100
- inputs: {"age_days": 5803, "days_push": 153, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases, no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2613, forks 580 (observed 2026-08-28T04:07:04.490724+00:00)

## What it is
mrjob is a Python library for writing and running Hadoop Streaming MapReduce jobs, with support for Amazon EMR, Google Cloud Dataproc, self-managed Hadoop clusters, and local execution for testing. It also supports launching Spark jobs and handles multi-step jobs, environment setup, and error log interpretation.

## Use cases
- run mapreduce jobs on hadoop from python
- run batch data processing on amazon emr
- test hadoop streaming jobs locally before deploying
- run multi-step mapreduce pipelines
- launch spark jobs on emr from python
- process large log files with mapreduce
- run word count and frequency analysis on a hadoop cluster

## When to choose
- you need to write Hadoop Streaming jobs in pure Python without JVM boilerplate
- you want to run the same job locally, on EMR, Dataproc, or your own Hadoop cluster
- you need multi-step MapReduce pipelines with automatic config and log interpretation

## When to avoid
- you are starting a new project - modern data processing tools like Spark, Dask, or Ray are better supported
- you need real-time or stream processing rather than batch MapReduce
- you are not tied to Hadoop or EMR infrastructure

## Facets
- artifact type: library
- maturity: maintenance
- function: etl, streaming, data-science, cli
- domain: big-data, cloud-computing, developer-tools
- platform: python, cli, cloud
- tags: mapreduce, hadoop, hadoop-streaming, amazon-emr, google-cloud-dataproc, spark, distributed-computing, batch-processing, data-engineering

## Member repositories
- Yelp/mrjob (main) score 66

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:07:04.490724+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T02:20:38.103887+00:00, confidence not recorded.
  - readme: https://github.com/Yelp/mrjob (fetched 2026-08-28T04:07:04.490724+00:00, sha d1cd578f9851)
  - registry_pypi: https://pypi.org/pypi/mrjob/json (fetched 2026-08-29T10:03:34.009440+00:00, sha 4bf39c634e73)
- Data as of 2026-08-30T08:39:29.467469+00:00.