OSU-NLP-Group/Mind2Web resource

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1021 stars · 124 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Mind2Web is the first dataset and benchmark for developing and evaluating generalist LLM-based web agents that follow natural language instructions to complete complex tasks on real-world websites. It contains over 2,000 open-ended tasks from 137 websites across 31 domains, with crowdsourced action sequences, plus fine-tuning code and models for element ranking and action prediction.

Use cases

• evaluate generalist web agents on real-world websites
• train LLM-based web navigation agents
• benchmark element ranking and action prediction on web pages
• research cross-domain generalization of web agents
• build agents that follow language instructions on any website
• compare web agent performance with macro and micro accuracy metrics

When to choose

• you need a large-scale, diverse benchmark for web agents using real websites rather than simulated ones
• you want to train or fine-tune models for web task automation from language instructions
• you need crowdsourced human action sequences as supervision for web navigation research
• you want paired HTML and screenshot data for multimodal web agent research

When to avoid

• you need a live interactive online evaluation environment rather than offline traces (see Online-Mind2Web or SeeAct instead)
• you only target a small set of specific websites or a simulated sandbox environment
• you need production-ready web automation rather than research benchmarking
• you cannot access the encrypted test set or agree to data contamination prevention terms

Facets

dataset · maturity active

machine-learning agent-framework rag data-science benchmarking nlp artificial-intelligence large-language-models machine-learning web-development data-science python cross-platform web-agent benchmark llm-agent dataset neurips-2023 web-navigation html-understanding huggingface jupyter-notebook academic-research task-automation element-ranking action-prediction ai-agents natural-language-processing research web-server gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="OSU-NLP-Group/Mind2Web")