beir-cellar/beir resource

Health v2 · maintenance only

Adoption not part of the score

2275 stars · 251 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

BEIR is a heterogeneous benchmark for information retrieval, aggregating 15+ diverse IR datasets with a common evaluation framework for NLP-based retrieval models. It enables zero-shot evaluation of models like BERT, DPR, ColBERT, and sentence-transformers across tasks.

Use cases

• evaluate my retrieval model across multiple IR datasets
• benchmark zero-shot passage retrieval models
• compare dense retrieval models like DPR and ColBERT
• find datasets for training and evaluating RAG retrievers
• measure how well my sentence-transformers model retrieves passages
• run a reproducible information retrieval leaderboard evaluation

When to choose

• you need standardized, comparable evaluation of retrieval models across diverse domains
• you are doing research on zero-shot or dense retrieval
• you want ready-made IR datasets with a common evaluation harness

When to avoid

• you need a production search engine rather than an evaluation benchmark
• your task is not text retrieval (e.g., image or multimodal retrieval)
• you need datasets outside the benchmark's covered IR tasks

Facets

dataset · maturity active

benchmarking search-engine nlp machine-learning rag machine-learning data-science python information-retrieval evaluation-framework zero-shot-retrieval sentence-transformers dense-retrieval passage-retrieval pytorch hugging-face natural-language-processing search retrieval-augmented-generation

Member repositories

For agents

markdown · JSON · MCP: product_card(name="beir-cellar/beir")