# huggingface/evaluate

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

Repository: https://github.com/huggingface/evaluate
Canonical: https://ross.abutalabs.com/products/evaluate
Homepage: https://huggingface.co/docs/evaluate
Language: Python
License: Apache-2.0
License Family: permissive
Topics: evaluation, machine-learning
Last push: 2026-07-06T13:22:00+00:00

## Health v2 (maintenance only)
Score: 74/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 91, release rhythm 36, longevity 100
- inputs: {"age_days": 1617, "days_push": 58, "days_rel": 349, "gap_med": 69, "n_releases_24m": 4}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2477, forks 336 (observed 2026-08-28T04:06:55.127032+00:00)

## What it is
Hugging Face's library for easily evaluating machine learning models and datasets with dozens of standardized metrics, comparisons, and measurements. It supports NLP, computer vision, and audio tasks across frameworks like PyTorch, TensorFlow, and JAX, and lets users share custom evaluation modules on the Hub.

## Use cases
- compute accuracy or BLEU scores for my model predictions
- evaluate a hugging face transformer model on a dataset
- compare performance of two machine learning models
- find standard metrics for machine translation evaluation
- measure properties of a dataset before training
- share a custom evaluation metric on the hugging face hub
- evaluate text classification and question answering pipelines

## When to choose
- you need standardized, well-tested ML metrics across NLP, vision, and audio tasks
- you work in the Hugging Face ecosystem and want Hub-integrated evaluation modules
- you want framework-agnostic metrics usable with NumPy, PyTorch, TensorFlow, or JAX
- you need an Evaluator to run model + dataset + metric pipelines end to end

## When to avoid
- you specifically need LLM evaluation - Hugging Face recommends the newer LightEval library instead
- you need a general-purpose unit testing framework rather than ML metrics
- you require cutting-edge actively developed features, since this library is in maintenance mode

## Facets
- artifact type: library
- maturity: maintenance
- function: machine-learning, benchmarking, testing, data-science
- domain: machine-learning, computer-vision, data-science
- platform: python, cross-platform
- tags: evaluation-metrics, hugging-face, model-evaluation, metrics, nlp-metrics, dataset-evaluation, natural-language-processing

## Member repositories
- huggingface/evaluate (main) score 74

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:06:55.127032+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T02:28:20.051005+00:00, confidence not recorded.
  - readme: https://github.com/huggingface/evaluate (fetched 2026-08-28T04:06:55.127032+00:00, sha f6e4ff456c83)
  - homepage: https://huggingface.co/docs/evaluate (fetched 2026-08-29T10:10:41.064134+00:00, sha 75d9be36f82e)
  - site_page: https://huggingface.co/docs (fetched 2026-08-29T10:10:41.073320+00:00, sha bdec26667b98)
  - site_page: https://huggingface.co/docs/evaluate/installation (fetched 2026-08-29T10:10:41.077694+00:00, sha 7e348014eea3)
  - site_page: https://huggingface.co/docs/evaluate/a_quick_tour (fetched 2026-08-29T10:10:41.079469+00:00, sha 9706b338b725)
  - site_page: https://huggingface.co/docs/evaluate/choosing_a_metric (fetched 2026-08-29T10:10:41.081410+00:00, sha 4cf4981d034a)
  - site_page: https://huggingface.co/docs/evaluate/creating_and_sharing (fetched 2026-08-29T10:10:41.083215+00:00, sha fe893bd76400)
  - site_page: https://huggingface.co/docs/evaluate/base_evaluator (fetched 2026-08-29T10:10:41.084927+00:00, sha 4b825a60658a)
  - site_page: https://huggingface.co/docs/evaluate/custom_evaluator (fetched 2026-08-29T10:10:41.086745+00:00, sha f9b950632230)
  - registry_pypi: https://pypi.org/pypi/evaluate/json (fetched 2026-08-29T10:10:41.088248+00:00, sha 0f439be2d72c)
- Data as of 2026-08-30T08:39:29.467469+00:00.