FranxYao/chain-of-thought-hub resource

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

2775 stars · 144 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A benchmarking hub and collection of evaluation scripts measuring large language models' complex reasoning ability using chain-of-thought prompting across tasks like GSM8K, MATH, BBH, MMLU, and HumanEval. It accompanies an academic paper and tracks model performance leaderboards over time.

Use cases

• benchmark llm reasoning ability
• compare chain-of-thought prompting performance across models
• evaluate a model on gsm8k and math datasets
• track which llm is best at complex reasoning
• find chain-of-thought prompt examples for evaluation
• measure long-context reasoning of large language models

When to choose

• you need standardized complex-reasoning benchmarks for LLMs
• you want curated chain-of-thought prompts and eval scripts
• you are comparing open and closed models on math, coding, or knowledge tasks

When to avoid

• you need a production inference or serving tool
• you want general-purpose LLM evaluation beyond reasoning tasks
• you need actively maintained benchmarks with the newest datasets

Facets

dataset · maturity maintenance

benchmarking prompt-engineering machine-learning large-language-models artificial-intelligence tutorials python chain-of-thought llm-evaluation reasoning-benchmarks gsm8k math-benchmark jupyter-notebooks natural-language-processing

Member repositories

For agents

markdown · JSON · MCP: product_card(name="FranxYao/chain-of-thought-hub")