Datasets › Mind2Web

Mind2Web

Introduced by Xiang Deng et al. in Mind2Web: Towards a Generalist Agent for the Web9 Jun 2023 archive 2025-07-28

Dataset Summary

Mind2Web is a dataset for developing and evaluating generalist agents for the web that can follow language instructions to complete complex tasks on any website. Existing datasets for web agents either use simulated websites or only cover a limited set of websites and tasks, thus not suitable for generalist web agents. With over 2,000 open-ended tasks collected from 137 websites spanning 31 domains and crowdsourced action sequences for the tasks, Mind2Web provides three necessary ingredients for building generalist web agents: 1. diverse domains, websites, and tasks, 2. use of real-world websites instead of simulated and simplified ones, and 3. a broad spectrum of user interaction patterns. Dataset Structure Data Fields

"annotation_id" (str): unique id for each task
"website" (str): website name
"domain" (str): website domain
"subdomain" (str): website subdomain
"confirmed_task" (str): task description
"action_reprs" (list[str]): human readable string representation of the action sequence
"actions" (list[dict]): list of actions (steps) to complete the task
    "action_uid" (str): unique id for each action (step)
    "raw_html" (str): raw html of the page before the action is performed
    "cleaned_html" (str): cleaned html of the page before the action is performed
    "operation" (dict): operation to perform
        "op" (str): operation type, one of CLICK, TYPE, SELECT
        "original_op" (str): original operation type, contain additional HOVER and ENTER that are mapped to CLICK, not used
        "value" (str): optional value for the operation, e.g., text to type, option to select
    "pos_candidates" (list[dict]): ground truth elements. Here we only include positive elements that exist in "cleaned_html" after our preprocessing, so "pos_candidates" might be empty. The original labeled element can always be found in the "raw_html".
        "tag" (str): tag of the element
        "is_original_target" (bool): whether the element is the original target labeled by the annotator
        "is_top_level_target" (bool): whether the element is a top level target find by our algorithm. please see the paper for more details.
        "backend_node_id" (str): unique id for the element
        "attributes" (str): serialized attributes of the element, use json.loads to convert back to dict
    "neg_candidates" (list[dict]): other candidate elements in the page after preprocessing, has similar structure as "pos_candidates"

Data Splits

train: 1,009 instances
test: (To prevent potential data leakage, please check our repo for information on obtaining the test set.)
    Cross Task: 252 instances, tasks from the same website are seen during training
    Cross Website: 177 instances, websites are not seen during training
    Cross Domain: 9,12 instances, entire domains are not seen during training

Benchmarks archive 2025-07-28

All 1 leaderboard whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

Syntology 6 newer papers were judged to report on this dataset's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 0 placed): task not recorded in the archive · Mind2Web (6 judged, 0 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).

First row (archive order)PaperCode
Mind2Web HTML-T5-XL Element Accuracy 73 A Real-World WebAgent with Planning, Long Context... — 1 Compare

Papers archive 2025-07-28

papers with a benchmark row: 1 · with a code link: 0 · where Syntology ran a sample: 0 (0 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology

Show: all papers with a benchmark rowonly where code ran: none of the 1 papers with a benchmark row has a Syntology run on record

Page 1 of 1: paper 1 of 1 with a leaderboard row on this dataset's benchmarks, newest first by the archive's date (ties by slug; undated papers last).

The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset, not that list; the archive's count for this dataset is 110. The Syntology column is from Syntology's graph, stated per sample; it is not part of any archive number. A line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified” (C is a part of N, never taken away from it); the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. When the archive marks a repository official for the paper, the cell starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record.

PaperCodeResultsDateSamples run Syntology
A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis 0 1 24 Jul 2023 not harvested

Dataset loaders archive 2025-07-28

No loader listed in the archive.

Tasks archive 2025-07-28

License archive 2025-07-28

CC BY 4.0 Deed

Modalities archive 2025-07-28

Languages archive 2025-07-28

Variants archive 2025-07-28

  • Mind2Web

1 variant name, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections