# beir-cellar/beir

A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.

Repository: https://github.com/beir-cellar/beir
Canonical: https://ross.abutalabs.com/products/beir
Homepage: http://beir.ai
Language: Python
License: Apache-2.0
License Family: permissive
Topics: nlp, information-retrieval, bert, benchmark, sentence-transformers, question-generation, retrieval, passage-retrieval, elasticsearch, dpr, sbert, retrieval-models, dataset, colbert, zero-shot-retrieval, deep-learning, pytorch, llm, rag
Last push: 2025-10-16T06:38:03+00:00

## Health v2 (maintenance only)
Score: 47/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 47, release rhythm 16, longevity 100
- inputs: {"age_days": 2053, "days_push": 321, "days_rel": 455, "gap_med": 98, "n_releases_24m": 2}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2275, forks 251 (observed 2026-08-28T04:06:33.440003+00:00)

## What it is
BEIR is a heterogeneous benchmark for information retrieval, aggregating 15+ diverse IR datasets with a common evaluation framework for NLP-based retrieval models. It enables zero-shot evaluation of models like BERT, DPR, ColBERT, and sentence-transformers across tasks.

## Use cases
- evaluate my retrieval model across multiple IR datasets
- benchmark zero-shot passage retrieval models
- compare dense retrieval models like DPR and ColBERT
- find datasets for training and evaluating RAG retrievers
- measure how well my sentence-transformers model retrieves passages
- run a reproducible information retrieval leaderboard evaluation

## When to choose
- you need standardized, comparable evaluation of retrieval models across diverse domains
- you are doing research on zero-shot or dense retrieval
- you want ready-made IR datasets with a common evaluation harness

## When to avoid
- you need a production search engine rather than an evaluation benchmark
- your task is not text retrieval (e.g., image or multimodal retrieval)
- you need datasets outside the benchmark's covered IR tasks

## Facets
- artifact type: dataset
- maturity: active
- function: benchmarking, search-engine, nlp, machine-learning, rag
- domain: machine-learning, data-science
- platform: python
- tags: information-retrieval, evaluation-framework, zero-shot-retrieval, sentence-transformers, dense-retrieval, passage-retrieval, pytorch, hugging-face, natural-language-processing, search, retrieval-augmented-generation

## Member repositories
- beir-cellar/beir (main) score 47

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:06:33.440003+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T02:41:28.933761+00:00, confidence not recorded.
  - readme: https://github.com/beir-cellar/beir (fetched 2026-08-28T04:06:33.440003+00:00, sha 3d5d7958f808)
  - homepage: http://beir.ai (fetched 2026-08-29T10:21:58.127020+00:00, sha 7d89af3e2aed)
  - registry_pypi: https://pypi.org/pypi/beir/json (fetched 2026-08-29T10:21:58.130161+00:00, sha 12b94d9df555)
- Data as of 2026-08-30T08:39:29.467469+00:00.