tau-bench resource

Health v2 · maintenance only

Adoption not part of the score

1882 stars · 472 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

τ-Bench (tau2-bench) is a Python benchmark for evaluating LLM agents on tool-agent-user interaction in real-world domains like retail, airline, telecom, and banking. It scores agents on conversations with simulated users, tool calls, knowledge retrieval, and policy adherence, including voice full-duplex evaluation, with a live public leaderboard.

Use cases

• benchmark my llm agent on tool calling and customer service tasks
• evaluate how reliably an agent follows domain policies
• compare models on multi-turn conversational agent tasks
• test voice agents on real-time full-duplex conversations
• evaluate rag pipelines on knowledge-intensive banking tasks
• measure pass^k reliability of agent trajectories
• reproduce published agent benchmark scores

When to choose

• you need standardized, verifiable evaluation of tool-using conversational agents
• you want to compare your model against a public leaderboard
• you need both text and voice agent evaluation with realistic simulated users

When to avoid

• you need a general-purpose agent framework for building production agents rather than evaluating them
• your domain is unrelated to the provided retail/airline/telecom/banking scenarios and you cannot author custom tasks
• you need cheap, fast evals - simulated user LLM calls add cost and latency

Facets

dataset · maturity active

benchmarking agent-framework llm-inference rag speech-recognition testing artificial-intelligence large-language-models chatbots developer-tools tutorials python cli cross-platform llm-benchmark tool-use agent-evaluation conversational-agents voice-agents leaderboard pass-k-metric customer-service-domains ai-agents natural-language-processing

Member repositories

For agents

markdown · JSON · MCP: product_card(name="sierra-research/tau2-bench")