harveyai/harvey-labs resource
A benchmark built to evaluate and improve agent capabilities for supporting legal work. observed · 2026-08-28
Health v2 · maintenance only
59/100
- Activity 99
- Release rhythm 35
- Longevity 11
Flags: no_releases young
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-09-03. Adoption (stars, forks) is never an input.
- gap_med: n/a
- age_days: 156
- days_rel: n/a
- days_push: 7
- n_releases_24m: 0
Adoption not part of the score
1261 stars · 215 forks observed · 2026-08-28
What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded
Harvey LAB (Legal Agent Benchmark) is an open-source benchmark from Harvey AI for evaluating LLM agents on realistic legal work, spanning 24+ practice areas with over 1,600 tasks. It pairs a dataset of agent instructions, documents, and scoring rubrics with a Python execution harness that runs agents, scores them via all-pass rubric and LLM-judge evaluation, and produces comparison reports.
Use cases
- benchmark LLM agents on legal tasks
- evaluate how well AI agents handle legal work like M&A due diligence
- find a dataset of legal tasks with rubrics for agent evaluation
- compare different models on legal reasoning and contract analysis
- run rubric-based scoring of agent outputs with an LLM judge
- measure agent capabilities for law firm workflows
When to choose
- You need a standardized, citable benchmark to measure how LLM agents perform on realistic legal assignments across many practice areas
- You want an execution harness with all-pass rubric scoring and LLM judge behavior for evaluating agent runs
- You are researching legal-domain agent capabilities and want versioned, reproducible evaluation tasks and reports
When to avoid
- You need a production legal AI assistant or document analysis product rather than an evaluation benchmark
- You want an agent framework for building or deploying agents rather than scoring them
- Your evaluation target is not legal work, in which case a general-purpose agent benchmark would fit better
Facets
dataset · maturity active
benchmarking machine-learning artificial-intelligence large-language-models legal python cli llm-evaluation agent-benchmark legal-tech rubric-scoring llm-as-judge evaluation-harness task-dataset ai-agents
5 sources
- readme: https://github.com/harveyai/harvey-labs · fetched 2026-08-28 · b24296a88638
- homepage: https://www.harvey.ai/blog/introducing-harveys-legal-agent-benchmark · fetched 2026-08-29 · f1e787854fe6
- site_page: https://www.harvey.ai/en-US/about/law-schools · fetched 2026-08-29 · fa5f1bfac59b
- site_page: https://www.harvey.ai/en-US/company · fetched 2026-08-29 · 776e73fb3f6b
- site_page: https://www.harvey.ai/en-US/newsroom · fetched 2026-08-29 · eb0c00a8744b
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| harveyai/harvey-labs | main | 59 |
For agents
markdown · JSON · MCP: product_card(name="harveyai/harvey-labs")
Data as of 2026-08-30T08:39:29.467469+00:00 · Report a problem