LiveBench/LiveBench resource

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

1294 stars · 119 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

LiveBench is a contamination-free benchmark for large language models that releases new questions monthly, drawn from recent datasets, papers, news, and IMDb synopses. It includes 18 diverse tasks across 6 categories with objective, verifiable ground-truth answers that are scored automatically without an LLM judge.

Use cases

• benchmark llm performance without test set contamination
• evaluate my model on a public leaderboard
• score llm answers objectively without an llm judge
• run monthly fresh benchmark questions for chat models
• compare api models on coding math and instruction following tasks
• evaluate agentic coding questions in docker

When to choose

• you need contamination-resistant evaluation of LLMs
• you want automatic objective scoring without judge models
• you want to submit a model to a public leaderboard

When to avoid

• you need local model inference support, which is unmaintained
• you need a static benchmark with a fixed question set
• you need subjective or open-ended evaluation criteria

Facets

dataset · maturity active

benchmarking llm-inference testing large-language-models machine-learning artificial-intelligence python cli llm-benchmark contamination-free evaluation leaderboard model-evaluation docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="LiveBench/LiveBench")