# evalplus/evalplus

Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024

Repository: https://github.com/evalplus/evalplus
Canonical: https://ross.abutalabs.com/products/evalplus
Homepage: https://evalplus.github.io
Language: Python
License: Apache-2.0
License Family: permissive
Topics: benchmark, chatgpt, gpt-4, large-language-models, program-synthesis, testing, efficiency
Last push: 2025-10-02T22:56:38+00:00

## Health v2 (maintenance only)
Score: 41/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 45, release rhythm 8, longevity 88
- inputs: {"age_days": 1236, "days_push": 335, "days_rel": 682, "gap_med": null, "n_releases_24m": 1}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1804, forks 206 (observed 2026-08-28T04:05:38.927357+00:00)

## What it is
EvalPlus is a Python framework and benchmark suite for rigorously evaluating LLM-synthesized code, providing HumanEval+ and MBPP+ datasets with massively extended test suites plus EvalPerf for code efficiency evaluation. It supports many LLM backends (OpenAI, Anthropic, Gemini, local models) and is widely used by major LLM teams.

## Use cases
- evaluate code generation quality of an LLM
- benchmark my model on HumanEval+ and MBPP+
- measure pass@1 for LLM-synthesized code
- compare code efficiency of LLM outputs
- run rigorous code correctness tests on generated Python
- evaluate long-context code understanding of models

## When to choose
- you need rigorous, test-augmented evaluation of LLM code generation
- you want standardized benchmarks recognized by major LLM teams
- you need both correctness and efficiency evaluation of generated code

## When to avoid
- you need to evaluate natural-language tasks rather than code
- you want a general-purpose unit testing framework for your own codebase
- you need production monitoring of live LLM applications

## Facets
- artifact type: library
- maturity: active
- function: testing, benchmarking, llm-inference, data-generation
- domain: large-language-models, machine-learning, developer-tools, testing
- platform: python, cli
- tags: code-evaluation, humaneval-plus, mbpp-plus, evalperf, program-synthesis, llm-benchmark, code-generation, leaderboard, docker, linux, macos

## Member repositories
- evalplus/evalplus (main) score 41

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:38.927357+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T03:21:48.886461+00:00, confidence not recorded.
  - readme: https://github.com/evalplus/evalplus (fetched 2026-08-28T04:05:38.927357+00:00, sha 3e347ad34b05)
  - homepage: https://evalplus.github.io (fetched 2026-08-29T11:00:46.231266+00:00, sha 08a87169adb9)
  - registry_pypi: https://pypi.org/pypi/evalplus/json (fetched 2026-08-29T11:00:46.240025+00:00, sha c13fbdae427c)
- Data as of 2026-08-30T08:39:29.467469+00:00.