{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/eia-environmental-injection-attack-on","title":"EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage","arxiv_id":"2409.11295","date":"2024-09-17","proceeding":null,"authors":["Zeyi Liao","Lingbo Mo","Chejian Xu","Mintong Kang","Jiawei Zhang","Chaowei Xiao","Yuan Tian","Bo Li","Huan Sun"],"abstract":"Generalist web agents have demonstrated remarkable potential in autonomously completing a wide range of tasks on real websites, significantly boosting human productivity. However, web tasks, such as booking flights, usually involve users' PII, which may be exposed to potential privacy risks if web agents accidentally interact with compromised websites, a scenario that remains largely unexplored in the literature. In this work, we narrow this gap by conducting the first study on the privacy risks of generalist web agents in adversarial environments. First, we present a realistic threat model for attacks on the website, where we consider two adversarial targets: stealing users' specific PII or the entire user request. Then, we propose a novel attack method, termed Environmental Injection Attack (EIA). EIA injects malicious content designed to adapt well to environments where the agents operate and our work instantiates EIA specifically for privacy scenarios in web environments. We collect 177 action steps that involve diverse PII categories on realistic websites from the Mind2Web, and conduct experiments using one of the most capable generalist web agent frameworks to date. The results demonstrate that EIA achieves up to 70% ASR in stealing specific PII and 16% ASR for full user request. Additionally, by accessing the stealthiness and experimenting with a defensive system prompt, we indicate that EIA is hard to detect and mitigate. Notably, attacks that are not well adapted for a webpage can be detected via human inspection, leading to our discussion about the trade-off between security and autonomy. However, extra attackers' efforts can make EIA seamlessly adapted, rendering such supervision ineffective. Thus, we further discuss the defenses at the pre- and post-deployment stages of the websites without relying on human supervision and call for more advanced defense strategies.","url_abs":"https://arxiv.org/abs/2409.11295v5","url_pdf":"https://arxiv.org/pdf/2409.11295v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"eia-environmental-injection-attack-on","repo_url":"https://github.com/osu-nlp-group/eia_against_webagent","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2409.11295","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.11295"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/osu-nlp-group/eia_against_webagent","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":12,"ran_honours":1,"unverified":1},"by_repo_kind":{"official":{"samples":14,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0e07e2733eab1746","entry":"clean_text","repo":"osu-nlp-group/eia_against_webagent","repo_kind":"official","path":"SeeAct/src/data_utils/dom_utils.py","file_url":"https://github.com/osu-nlp-group/eia_against_webagent/blob/HEAD/SeeAct/src/data_utils/dom_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0e07e2733eab1746"}},{"code_sha256_prefix":"b64eb9b4e7d05784","entry":"extract_elements_by_ids","repo":"osu-nlp-group/eia_against_webagent","repo_kind":"official","path":"SeeAct/src/data_utils/image_utils.py","file_url":"https://github.com/osu-nlp-group/eia_against_webagent/blob/HEAD/SeeAct/src/data_utils/image_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b64eb9b4e7d05784"}},{"code_sha256_prefix":"9fb9dab3374bb263","entry":"extract_topk_elements","repo":"osu-nlp-group/eia_against_webagent","repo_kind":"official","path":"SeeAct/src/data_utils/image_utils.py","file_url":"https://github.com/osu-nlp-group/eia_against_webagent/blob/HEAD/SeeAct/src/data_utils/image_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9fb9dab3374bb263"}},{"code_sha256_prefix":"3db599f0f49fee4f","entry":"generate_new_query_prompt","repo":"osu-nlp-group/eia_against_webagent","repo_kind":"official","path":"SeeAct/src/data_utils/format_prompt_utils.py","file_url":"https://github.com/osu-nlp-group/eia_against_webagent/blob/HEAD/SeeAct/src/data_utils/format_prompt_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3db599f0f49fee4f"}},{"code_sha256_prefix":"dcac82655df6139c","entry":"generate_query_prompt","repo":"osu-nlp-group/eia_against_webagent","repo_kind":"official","path":"SeeAct/src/dy_human_dynamic.py","file_url":"https://github.com/osu-nlp-group/eia_against_webagent/blob/HEAD/SeeAct/src/dy_human_dynamic.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dcac82655df6139c"}},{"code_sha256_prefix":"061acc9fbe79bb1c","entry":"generate_query_prompt","repo":"osu-nlp-group/eia_against_webagent","repo_kind":"official","path":"SeeAct/src/data_utils/format_prompt_utils.py","file_url":"https://github.com/osu-nlp-group/eia_against_webagent/blob/HEAD/SeeAct/src/data_utils/format_prompt_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"061acc9fbe79bb1c"}},{"code_sha256_prefix":"56fc0416b2cf42ae","entry":"get_descendants","repo":"osu-nlp-group/eia_against_webagent","repo_kind":"official","path":"SeeAct/src/data_utils/dom_utils.py","file_url":"https://github.com/osu-nlp-group/eia_against_webagent/blob/HEAD/SeeAct/src/data_utils/dom_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"56fc0416b2cf42ae"}},{"code_sha256_prefix":"b6a99db34d8f1684","entry":"init_cdp_session","repo":"osu-nlp-group/eia_against_webagent","repo_kind":"official","path":"SeeAct/src/dy_human_dynamic.py","file_url":"https://github.com/osu-nlp-group/eia_against_webagent/blob/HEAD/SeeAct/src/dy_human_dynamic.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b6a99db34d8f1684"}},{"code_sha256_prefix":"f9d1a77f72c2e2f9","entry":"normal_new_context_async","repo":"osu-nlp-group/eia_against_webagent","repo_kind":"official","path":"SeeAct/src/demo_utils/browser_helper.py","file_url":"https://github.com/osu-nlp-group/eia_against_webagent/blob/HEAD/SeeAct/src/demo_utils/browser_helper.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9d1a77f72c2e2f9"}},{"code_sha256_prefix":"c0e8041ef4fac1b2","entry":"process_attack_type_subtypes","repo":"osu-nlp-group/eia_against_webagent","repo_kind":"official","path":"injection/inject_malicious_form.py","file_url":"https://github.com/osu-nlp-group/eia_against_webagent/blob/HEAD/injection/inject_malicious_form.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c0e8041ef4fac1b2"}},{"code_sha256_prefix":"d6c9b5270beb73e5","entry":"remove_extra_eol","repo":"osu-nlp-group/eia_against_webagent","repo_kind":"official","path":"SeeAct/src/data_utils/dom_utils.py","file_url":"https://github.com/osu-nlp-group/eia_against_webagent/blob/HEAD/SeeAct/src/data_utils/dom_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d6c9b5270beb73e5"}},{"code_sha256_prefix":"5e4b02e5bc753357","entry":"similar","repo":"osu-nlp-group/eia_against_webagent","repo_kind":"official","path":"SeeAct/src/dy_human_dynamic.py","file_url":"https://github.com/osu-nlp-group/eia_against_webagent/blob/HEAD/SeeAct/src/dy_human_dynamic.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5e4b02e5bc753357"}},{"code_sha256_prefix":"ce51ce402f71333e","entry":"write_html_file","repo":"osu-nlp-group/eia_against_webagent","repo_kind":"official","path":"injection/inject_malicious_form.py","file_url":"https://github.com/osu-nlp-group/eia_against_webagent/blob/HEAD/injection/inject_malicious_form.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ce51ce402f71333e"}},{"code_sha256_prefix":"431150ea1b184bf1","entry":"generate_prompt","repo":"osu-nlp-group/eia_against_webagent","repo_kind":"official","path":"SeeAct/src/data_utils/prompts.py","file_url":"https://github.com/osu-nlp-group/eia_against_webagent/blob/HEAD/SeeAct/src/data_utils/prompts.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"431150ea1b184bf1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}