xlang-ai/OSWorld resource

Health v2 · maintenance only

Adoption not part of the score

3108 stars · 526 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

OSWorld is a benchmark and scalable real computer environment for evaluating multimodal agents on open-ended computer tasks across operating systems, with 369 real-world tasks and reproducible setup/evaluation scripts. It supports task setup, execution-based evaluation, and interactive learning in VMs hosted via VMware, VirtualBox, Docker, or cloud providers.

Use cases

• benchmark multimodal llm agents on real computer tasks
• evaluate gui agents in a real desktop environment
• compare computer-use agent performance across models
• set up an interactive environment for agent reinforcement learning
• test agents on tasks involving arbitrary desktop applications
• run reproducible evaluations of os automation agents

When to choose

• you need a standardized, reproducible benchmark for computer-use or GUI agents
• you want execution-based evaluation rather than static datasets
• you need a real OS environment spanning multiple applications for agent testing

When to avoid

• you only need a lightweight text-only agent benchmark without OS interaction
• you cannot provision virtual machines or cloud instances for evaluation
• you need a production agent framework rather than an evaluation benchmark

Facets

dataset · maturity active

benchmarking agent-framework testing artificial-intelligence large-language-models developer-tools python cloud multimodal-agents computer-use gui-agents evaluation-benchmark virtual-machines reinforcement-learning-environment ai-agents linux docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="xlang-ai/OSWorld")