# hendrycks/test

Measuring Massive Multitask Language Understanding | ICLR 2021

Repository: https://github.com/hendrycks/test
Canonical: https://ross.abutalabs.com/products/test
Homepage: https://arxiv.org/abs/2009.03300
Language: Python
License: MIT
License Family: permissive
Topics: muti-task, transfer-learning, gpt-3, few-shot-learning
Last push: 2023-05-28T18:28:58+00:00

## Health v2 (maintenance only)
Score: 32/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 0, release rhythm 35, longevity 100
- inputs: {"age_days": 2186, "days_push": 1193, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1613, forks 119 (observed 2026-08-28T04:05:11.231926+00:00)

## What it is
The MMLU (Massive Multitask Language Understanding) benchmark dataset and evaluation code from Hendrycks et al., ICLR 2021. It contains multiple-choice questions spanning 57 academic and professional subjects, plus OpenAI API evaluation scripts for measuring language model accuracy.

## Use cases
- evaluate an LLM on MMLU benchmark
- measure multitask knowledge of a language model
- compare my model's accuracy against GPT-3 leaderboard scores
- run few-shot evaluation of a text model on 57 subjects
- download MMLU test questions for model evaluation

## When to choose
- you need a standardized academic knowledge benchmark for LLM evaluation
- you want to compare your model against published MMLU leaderboard results
- you need multiple-choice evaluation data across humanities, STEM, and social sciences

## When to avoid
- you need a live leaderboard or modern harness — the repo's evaluation code is dated and most teams now use lm-evaluation-harness
- you need training data rather than an evaluation test set
- you need benchmarks beyond multiple-choice academic questions

## Facets
- artifact type: dataset
- maturity: maintenance
- function: machine-learning, benchmarking, nlp
- domain: large-language-models, machine-learning, artificial-intelligence, education
- platform: python
- tags: mmlu, llm-evaluation, few-shot-learning, benchmark-dataset, gpt-3, multiple-choice

## Member repositories
- hendrycks/test (main) score 32

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:11.231926+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T03:51:06.020123+00:00, confidence not recorded.
  - readme: https://github.com/hendrycks/test (fetched 2026-08-28T04:05:11.231926+00:00, sha 1b64d2d6b999)
  - homepage: https://arxiv.org/abs/2009.03300 (fetched 2026-08-29T11:22:56.737420+00:00, sha 31bbb53861b8)
  - site_page: https://info.arxiv.org/about/donate.html (fetched 2026-08-29T11:22:56.747221+00:00, sha cca9c3a11c56)
  - site_page: https://info.arxiv.org/about/ourmembers.html (fetched 2026-08-29T11:22:56.751545+00:00, sha 47cbc55ff1de)
  - site_page: https://info.arxiv.org/about (fetched 2026-08-29T11:22:56.753366+00:00, sha a1f16f915a9a)
  - site_page: https://info.arxiv.org/labs/index.html (fetched 2026-08-29T11:22:56.749732+00:00, sha b14a8d05a0ec)
- Data as of 2026-08-30T08:39:29.467469+00:00.