pinchbench/skill resource

Health v2 · maintenance only

Adoption not part of the score

1325 stars · 154 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

PinchBench is a benchmarking system that evaluates LLM models as OpenClaw coding agents using 53 real-world tasks like scheduling, coding, research, and email triage. This repository contains the benchmark skill/task definitions, with results published on a public leaderboard at pinchbench.com.

Use cases

• compare LLM models for coding agent performance
• evaluate which model is best for my AI agent
• benchmark LLM tool usage and multi-step reasoning
• find the best value or cheapest model for agent tasks
• run real-world agent tasks against different models
• measure LLM success rate, speed, and cost
• add custom benchmark tasks for AI agents

When to choose

• You run OpenClaw (or similar) agents and need to pick the best model
• You want real-world task benchmarks rather than synthetic LLM tests
• You need cost/speed/success-rate comparisons across many models via OpenRouter

When to avoid

• You need standard academic benchmarks like SWE-bench or HumanEval
• You don't use OpenClaw or an equivalent agent runtime
• You want to benchmark non-agent capabilities like raw code completion

Facets

dataset · maturity active

benchmarking llm-inference agent-framework testing data-generation large-language-models machine-learning developer-tools testing python cli cross-platform llm-benchmark coding-agents leaderboard openclaw task-suite llm-judge evaluation ai-agents

Member repositories

For agents

markdown · JSON · MCP: product_card(name="pinchbench/skill")