sunny-glow/Auto-BenchMax

Health v2 · maintenance only

Flags: no_releases young no_license

Adoption not part of the score

1317 stars · 27 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Auto-BenchMax is a Python pipeline that automatically synthesizes benchmark-targeted training data for LLMs, claiming to more than double a base model's score on agentic benchmarks like MCP-Atlas and Tau2-bench. It ships a data-construction pipeline, training scripts, an evaluation environment, and a skill that lets agent tools like Claude Code drive synthesis for custom tool-use benchmarks.

Use cases

• synthesize training data for an agentic benchmark
• improve my model's score on MCP-Atlas
• generate fine-tuning data targeted at a tool-use benchmark
• reproduce benchmark score improvements with one click
• create LLM-judged or rule-based benchmark training sets
• fine-tune a model to double its benchmark baseline

When to choose

• you need to boost a model's score on a specific tool-use or agentic benchmark
• you want a reproducible data-synthesis plus training pipeline
• you use a skill-capable agent like Claude Code and want one-sentence automation

When to avoid

• you need genuinely general capability gains rather than benchmark-targeted optimization
• you lack the compute to fine-tune models
• you need a license-cleared project for commercial use, since no license is specified

Facets

library · maturity active

llm-training data-generation agent-framework machine-learning large-language-models machine-learning artificial-intelligence developer-tools python cli benchmark-optimization synthetic-data fine-tuning tool-use skill-based-pipeline llm-judge training-scripts ai-agents linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="sunny-glow/Auto-BenchMax")