OpenGenerativeAI/llm-colosseum

Health v2 · maintenance only

Adoption not part of the score

1482 stars · 182 forks observed · 2026-09-24

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

LLM Colosseum is a benchmarking tool that pits large language models against each other in real-time Street Fighter III matches to evaluate their decision-making quality. It produces an ELO-based leaderboard from hundreds of automated fights between models like GPT-4o, Pixtral, Claude, and Llama.

Use cases

• benchmark llms by playing street fighter
• compare llm decision making in real time
• elo ranking of large language models
• evaluate llm agents in a game environment
• test which ai model is the best fighter
• alternative llm benchmark beyond static tests

When to choose

• you want a novel, real-time evaluation of LLM reasoning and adaptability
• you want to compare vision and text modes of frontier models in an interactive setting
• you want an entertaining demo of LLM agent capabilities

When to avoid

• you need rigorous, reproducible academic benchmarks
• you need standard NLP task evaluation like MMLU or code generation
• you need a production LLM evaluation pipeline

Facets

application · maturity active

benchmarking llm-inference agent-framework machine-learning large-language-models artificial-intelligence gaming-tools analytics python browser llm-evaluation elo-ranking street-fighter game-benchmark reinforcement-learning-comparison jupyter-notebook web desktop

Member repositories

For agents

markdown · JSON · MCP: product_card(name="OpenGenerativeAI/llm-colosseum")