# microsoftarchive/promptbench

A unified evaluation framework for large language models

Repository: https://github.com/microsoftarchive/promptbench
Canonical: https://ross.abutalabs.com/products/promptbench
Homepage: http://aka.ms/promptbench
Language: Python
License: MIT
License Family: permissive
Topics: adversarial-attacks, chatgpt, evaluation, large-language-models, robustness, prompt, prompt-engineering, benchmark
Archived: true
Last push: 2026-02-20T18:55:32+00:00

## Health v2 (maintenance only)
Score: 10/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 68, release rhythm 35, longevity 84
- inputs: {"age_days": 1177, "days_push": 194, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases, archived
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2819, forks 221 (observed 2026-08-28T04:07:23.935679+00:00)

## What it is
PromptBench is a unified Python library from Microsoft for evaluating and understanding large language models across many datasets, models, and prompt techniques. It includes adversarial prompt attack modules and robustness analysis to measure how LLM performance degrades under perturbed prompts.

## Use cases
- benchmark llm performance across multiple datasets
- evaluate prompt robustness against adversarial attacks
- compare gpt-4o, gemini, mistral and open-source models
- measure how prompt wording affects model accuracy
- run multi-prompt evaluation efficiently
- evaluate multimodal llm capabilities

## When to choose
- you need a standardized harness to compare many LLMs on academic benchmarks
- you want to study prompt sensitivity and adversarial robustness of LLMs
- you need reproducible evaluation with support for major commercial and open models

## When to avoid
- you need a production serving or inference framework rather than evaluation
- you want lightweight ad-hoc testing of a single model without benchmark overhead
- you need a hosted leaderboard service rather than a local library

## Facets
- artifact type: framework
- maturity: maintenance
- function: benchmarking, llm-inference, prompt-engineering, testing
- domain: large-language-models, machine-learning, artificial-intelligence
- platform: python, cross-platform
- tags: llm-evaluation, adversarial-robustness, prompt-robustness, benchmarking-suite, microsoft

## Member repositories
- microsoftarchive/promptbench (main) score 10

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:07:23.935679+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T07:38:33.588509+00:00, confidence not recorded.
  - readme: https://github.com/microsoftarchive/promptbench (fetched 2026-08-28T04:07:23.935679+00:00, sha bda340c071b2)
- Data as of 2026-08-30T08:39:29.467469+00:00.