vectara/hallucination-leaderboard resource

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

3306 stars · 107 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

A public leaderboard maintained by Vectara that ranks LLMs by how often they hallucinate when summarizing short documents, computed with Vectara's HHEM hallucination evaluation model. It is updated regularly and also available as an interactive Hugging Face Space.

Use cases

• compare llms for hallucination rates
• find which llm is most factually consistent at summarization
• check hallucination rate of a model before choosing it
• benchmark llms on factual consistency
• evaluate which model to use for document summarization
• track how llm hallucination rates change over time

When to choose

• you need an up-to-date, third-party comparison of LLM factual consistency in summarization
• you are selecting a model for RAG or summarization pipelines where hallucination risk matters
• you want a simple ranked reference backed by a published evaluation model (HHEM)

When to avoid

• you need task-level evaluation beyond short-document summarization
• you require a runnable evaluation harness for your own datasets rather than a published ranking
• you need benchmarks of reasoning, coding, or other non-summarization capabilities

Facets

dataset · maturity active

benchmarking machine-learning nlp large-language-models machine-learning artificial-intelligence python llm-evaluation hallucination-detection leaderboard summarization factual-consistency hhem web

Member repositories

For agents

markdown · JSON · MCP: product_card(name="vectara/hallucination-leaderboard")