benchflow-ai/skillsbench resource

Health v2 · maintenance only

Adoption not part of the score

1725 stars · 364 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

SkillsBench is the first benchmark for evaluating how effectively AI agents use skills—modular folders of instructions, scripts, and resources—across expert-curated professional tasks. It provides sandboxed task packages with oracle solutions and outcome-based verifiers, runnable via the BenchFlow CLI on Modal or Docker.

Use cases

• benchmark how well AI agents use skills
• compare LLM performance with and without skills
• evaluate agent skill composition across tasks
• measure agent resolution rate and wall-clock efficiency
• create and contribute agent evaluation tasks
• run reproducible agent evaluations in cloud sandboxes

When to choose

• you need to measure how effectively agents leverage skills on specialized workflows
• you want to compare frontier models on skill-using agent tasks
• you need a gym-style evaluation harness with oracle sanity checks and verifiers
• you want to contribute or curate benchmark tasks for agent skills

When to avoid

• you need a general-purpose agent framework for building products rather than evaluation
• you want a simple unit-testing tool for code
• you cannot use Modal or Docker sandbox execution
• you need benchmarking of non-agent software performance

Facets

dataset · maturity active

benchmarking agent-framework testing artificial-intelligence large-language-models developer-tools tutorials python cli cloud agent-skills benchmark llm-evaluation skill-composition sandboxed-tasks leaderboard benchflow evaluation ai-agents docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="benchflow-ai/skillsbench")