# openai/mle-bench

MLE-bench is a benchmark for measuring how well AI agents perform at machine learning engineering

Repository: https://github.com/openai/mle-bench
Canonical: https://ross.abutalabs.com/products/mle-bench
Homepage: https://openai.com/index/mle-bench/
Language: Python
License: NOASSERTION
License Family: other
Last push: 2026-04-24T17:33:44+00:00

## Health v2 (maintenance only)
Score: 58/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 79, release rhythm 35, longevity 49
- inputs: {"age_days": 694, "days_push": 131, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases, no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1720, forks 257 (observed 2026-08-28T04:05:26.610836+00:00)

## What it is
MLE-bench is an open-source benchmark from OpenAI that measures how well AI agents perform machine learning engineering tasks, built from 75 Kaggle competitions with human baselines. It includes dataset construction code, evaluation/grading logic, and agent scaffolds for evaluating frontier LLMs.

## Use cases
- evaluate how well an LLM agent does machine learning engineering
- benchmark AI agents on Kaggle-style ML competitions
- compare agent scaffolds like AIDE on ML tasks
- measure whether an agent can train models and prepare datasets autonomously
- research contamination and resource scaling for ML agents
- reproduce the MLE-bench leaderboard results for a new agent

## When to choose
- you are researching or benchmarking LLM agents on real-world ML engineering tasks
- you want standardized Kaggle-derived tasks with human baselines and grading
- you need open-source evaluation harness code and reference agent scaffolds

## When to avoid
- you need a general coding benchmark like SWE-bench rather than ML-specific tasks
- you want a production tool for running ML pipelines rather than an evaluation benchmark
- you require an actively accepting leaderboard submissions (currently paused)

## Facets
- artifact type: dataset
- maturity: active
- function: benchmarking, agent-framework, machine-learning, testing
- domain: machine-learning, artificial-intelligence, developer-tools, tutorials
- platform: python, cross-platform
- tags: kaggle, evaluation, llm-agents, ml-engineering, leaderboard, openai, ai-agents, docker, linux

## Member repositories
- openai/mle-bench (main) score 58

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:26.610836+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T03:34:05.422167+00:00, confidence not recorded.
  - readme: https://github.com/openai/mle-bench (fetched 2026-08-28T04:05:26.610836+00:00, sha f75375122f4c)
  - homepage: https://openai.com/index/mle-bench/ (fetched 2026-08-29T11:09:57.388798+00:00, sha 688b58243c42)
  - site_page: https://openai.com/about (fetched 2026-08-29T11:09:57.398570+00:00, sha de4b627b33e3)
  - site_page: https://developers.openai.com/api/docs (fetched 2026-08-29T11:09:57.400826+00:00, sha d617c5215f2a)
  - site_page: https://developers.openai.com/ (fetched 2026-08-29T11:09:57.403130+00:00, sha 7d240c906018)
- Data as of 2026-08-30T08:39:29.467469+00:00.