# bird-bench/BIRD-Interact

[ICLR 2026 Oral] BIRD-INTERACT: Re-imagines Text-to-SQL evaluation via lens of dynamic interactions.

Repository: https://github.com/bird-bench/BIRD-Interact
Canonical: https://ross.abutalabs.com/products/bird-interact
Homepage: https://bird-interact.github.io/
Language: Python
License: MIT
License Family: permissive
Last push: 2026-03-29T07:54:22+00:00

## Health v2 (maintenance only)
Score: 52/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 74, release rhythm 35, longevity 33
- inputs: {"age_days": 469, "days_push": 157, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1011, forks 25 (observed 2026-08-28T04:03:13.117374+00:00)

## What it is
BIRD-INTERACT is an interactive Text-to-SQL benchmark that evaluates LLMs through dynamic multi-turn interactions with a simulated user, a hierarchical knowledge base, and database documentation. It provides 600 annotated tasks spanning BI and CRUD operations in passive conversational and active agentic modes, each guarded by executable test cases.

## Use cases
- evaluate LLMs on interactive text-to-sql tasks
- benchmark agentic database assistants
- test conversational SQL generation with clarifying questions
- measure model performance on CRUD and BI database tasks
- compare reasoning models on multi-turn database interaction
- research dynamic evaluation for text-to-sql

## When to choose
- you need a rigorous, executable-test-guarded benchmark for interactive Text-to-SQL
- you want to evaluate agents that must ask clarifying questions or act proactively
- you are benchmarking production-ready database assistant capabilities

## When to avoid
- you need a simple static text-to-SQL dataset without interaction
- you need a lightweight single-turn SQL generation benchmark
- your focus is non-SQL code generation evaluation

## Facets
- artifact type: dataset
- maturity: active
- function: benchmarking, llm-inference, agent-framework, database, nlp
- domain: databases, large-language-models, artificial-intelligence
- platform: python, cli
- tags: text-to-sql, benchmark, interactive-evaluation, user-simulator, llm-evaluation, sql, ai-agents, natural-language-processing

## Member repositories
- bird-bench/BIRD-Interact (main) score 52

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:03:13.117374+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T07:11:49.541259+00:00, confidence not recorded.
  - readme: https://github.com/bird-bench/BIRD-Interact (fetched 2026-08-28T04:03:13.117374+00:00, sha 369758f88880)
  - homepage: https://bird-interact.github.io/ (fetched 2026-08-29T13:11:38.750692+00:00, sha 4250d1f7a2d2)
- Data as of 2026-08-30T08:39:29.467469+00:00.