# mjpost/sacrebleu

Reference BLEU implementation that auto-downloads test sets and reports a version string to facilitate cross-lab comparisons

Repository: https://github.com/mjpost/sacrebleu
Canonical: https://ross.abutalabs.com/products/sacrebleu
Language: Python
License: Apache-2.0
License Family: permissive
Last push: 2026-08-20T07:32:06+00:00

## Health v2 (maintenance only)
Score: 76/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 98, release rhythm 33, longevity 100
- inputs: {"age_days": 3260, "days_push": 13, "days_rel": 233, "gap_med": 373, "n_releases_24m": 2}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1258, forks 177 (observed 2026-08-28T04:04:09.659119+00:00)

## What it is
SacreBLEU is a Python library and CLI tool for computing shareable, comparable, and reproducible BLEU, chrF, and TER scores for machine translation evaluation. It automatically downloads and manages standard WMT test sets and emits a version string so scores can be compared across papers and labs.

## Use cases
- compute BLEU scores for machine translation output
- score translations against WMT test sets without manual downloads
- compare BLEU scores reproducibly across research papers
- evaluate translations with chrF or TER metrics
- run statistical significance tests on MT evaluation results
- score detokenized Japanese or Chinese translations with appropriate tokenizers

## When to choose
- you need WMT-official BLEU scores that match mteval-v13a.pl
- you want automatic test set downloading and standardized tokenization
- you need reproducible, comparable MT evaluation metrics for publications

## When to avoid
- you need metrics beyond BLEU, chrF, and TER such as BERTScore or COMET
- you are evaluating tasks other than machine translation
- you need a general-purpose NLP toolkit rather than a scoring metric

## Facets
- artifact type: library
- maturity: active
- function: machine-learning, nlp, benchmarking, cli, testing
- domain: machine-learning, developer-tools
- platform: python, cli, cross-platform
- tags: bleu, machine-translation-evaluation, chrf, ter, test-set-management, computational-linguistics, wmt, natural-language-processing, algorithms

## Member repositories
- mjpost/sacrebleu (main) score 76

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:04:09.659119+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T05:05:36.161624+00:00, confidence not recorded.
  - readme: https://github.com/mjpost/sacrebleu (fetched 2026-08-28T04:04:09.659119+00:00, sha 1a72ee81932a)
  - registry_pypi: https://pypi.org/pypi/sacrebleu/json (fetched 2026-08-29T12:17:33.951910+00:00, sha e51e3d227188)
- Data as of 2026-08-30T08:39:29.467469+00:00.