mjpost/sacrebleu

Health v2 · maintenance only

Adoption not part of the score

1258 stars · 177 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

SacreBLEU is a Python library and CLI tool for computing shareable, comparable, and reproducible BLEU, chrF, and TER scores for machine translation evaluation. It automatically downloads and manages standard WMT test sets and emits a version string so scores can be compared across papers and labs.

Use cases

• compute BLEU scores for machine translation output
• score translations against WMT test sets without manual downloads
• compare BLEU scores reproducibly across research papers
• evaluate translations with chrF or TER metrics
• run statistical significance tests on MT evaluation results
• score detokenized Japanese or Chinese translations with appropriate tokenizers

When to choose

• you need WMT-official BLEU scores that match mteval-v13a.pl
• you want automatic test set downloading and standardized tokenization
• you need reproducible, comparable MT evaluation metrics for publications

When to avoid

• you need metrics beyond BLEU, chrF, and TER such as BERTScore or COMET
• you are evaluating tasks other than machine translation
• you need a general-purpose NLP toolkit rather than a scoring metric

Facets

library · maturity active

machine-learning nlp benchmarking cli testing machine-learning developer-tools python cli cross-platform bleu machine-translation-evaluation chrf ter test-set-management computational-linguistics wmt natural-language-processing algorithms

Member repositories

For agents

markdown · JSON · MCP: product_card(name="mjpost/sacrebleu")