harbor-framework/terminal-bench-1 resource

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

2554 stars · 568 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Terminal-Bench is a benchmark and execution harness for evaluating LLM agents on complex, real-world tasks in a terminal sandbox. It combines a dataset of ~100+ terminal tasks with tooling that connects language models to the environment for reproducible evaluation.

Use cases

• benchmark llm agents on terminal tasks
• evaluate how well a model can use a shell autonomously
• compare coding agents on end-to-end system tasks
• run reproducible agent evaluations in a sandbox
• contribute new challenging tasks for agent testing

When to choose

• you need a standardized, reproducible benchmark for terminal-based agent capabilities
• you are building or stress-testing LLM agents that operate in a shell
• you want to compare models on real-world system-level tasks like compiling code or configuring servers

When to avoid

• you need a general-purpose chat or reasoning benchmark rather than terminal tasks
• you want a lightweight unit-testing tool rather than an agent evaluation harness
• your agents operate in GUI or web environments rather than text terminals

Facets

dataset · maturity active

benchmarking agent-framework testing cli large-language-models developer-tools python cli llm-benchmark terminal-agents evaluation-harness agent-evaluation sandbox ai-agents command-line docker linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="harbor-framework/terminal-bench-1")