# Cloud-CV/EvalAI

:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI

Repository: https://github.com/Cloud-CV/EvalAI
Canonical: https://ross.abutalabs.com/products/evalai
Homepage: https://eval.ai
Language: Python
License: NOASSERTION
License Family: other
Topics: ai, machine-learning, django, angularjs, python, ai-challenges, docker, reproducible-research, reproducibility, evaluation, challenge, evalai, leaderboard, artificial-intelligence, codecov, coveralls, evaluation-framework, github-actions
Last push: 2026-08-23T23:56:42+00:00

## Health v2 (maintenance only)
Score: 67/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 8, longevity 100
- inputs: {"age_days": 3604, "days_push": 10, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2039, forks 983 (observed 2026-08-28T04:06:08.483822+00:00)

## What it is
EvalAI is an open-source platform for evaluating and comparing machine learning and AI algorithms at scale. It provides a central leaderboard and submission interface with remote, map-reduce-backed evaluation to make benchmark results reproducible.

## Use cases
- host an AI challenge with a public leaderboard
- evaluate ML model submissions against a test set
- reproduce benchmark results from research papers
- run remote evaluation of large-scale ML challenges
- compare algorithms on standardized dataset splits
- manage private and public leaderboards for a competition

## When to choose
- you need to host an ML/AI competition with submissions and leaderboards
- you want reproducible, standardized evaluation of algorithms at scale
- you need custom evaluation phases, splits, and metrics
- you want a self-hosted alternative to closed challenge platforms

## When to avoid
- you only need simple unit testing of ML code rather than challenge evaluation
- you want a lightweight single-model benchmark harness without a web platform
- you cannot operate a Django/Angular web service with Docker infrastructure

## Facets
- artifact type: service
- maturity: active
- function: machine-learning, benchmarking, api-framework, web-framework, self-hosted
- domain: machine-learning, artificial-intelligence, data-science, analytics, web-development
- platform: python, self-hosted, cross-platform
- tags: ai-challenges, leaderboard, evaluation-platform, reproducible-research, django, angularjs, ml-evaluation, competition-hosting, docker, web-server

## Member repositories
- Cloud-CV/EvalAI (main) score 67

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:06:08.483822+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T02:59:11.484333+00:00, confidence not recorded.
  - readme: https://github.com/Cloud-CV/EvalAI (fetched 2026-08-28T04:06:08.483822+00:00, sha 85fdf14b85c3)
  - homepage: https://eval.ai (fetched 2026-08-29T10:38:43.268765+00:00, sha 44136fa355b3)
- Data as of 2026-08-30T08:39:29.467469+00:00.