{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2606-05647","title":"Coding with \"Enemy\": Can Human Developers Detect AI Agent Sabotage?","arxiv_id":"2606.05647","date":"2026-06-04","proceeding":null,"authors":["Jingheng Ye","Huiqi Zou","Simon Yu","Weiyan Shi"],"abstract":"AI coding agents are increasingly embedded in real-world software development, collaborating with human developers while gaining broader access to codebases and tools. This creates a new attack surface: an agent can exploit human trust to sabotage development, for instance by inserting malicious code to accomplish a hidden side task. Most prior work studies AI sabotage in AI-only settings, paying limited attention to the role of human oversight in detecting and mitigating such malicious behavior. To address this gap, we conduct the first large-scale study of human oversight in AI coding sabotage. Over 100 participants collaborate with one of four frontier models (Claude-Opus-4.6, GPT-5.4, Gemini-3.1-Pro, and MiniMax-M2.7) on a long-horizon coding task lasting around five hours, designed to mimic real-world workflows. We find that 94% of developers fail to detect sabotage, and our analysis of participant feedback attributes this vulnerability to minimal code review, plausible cover story, and overtrust in agents. We further test the effectiveness of a safety monitor in one condition: while the monitor reduces sabotage success, 56% of participants still accept the malicious code, ignoring its warnings. Drawing on participant feedback, we offer actionable suggestions for better monitor design. This work complements existing AI safety research and highlights an urgent need for human-centric safety mechanisms that account for human factors, particularly in long-horizon, real-world development settings.","url_abs":"https://arxiv.org/abs/2606.05647","url_pdf":"https://arxiv.org/pdf/2606.05647","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2606.05647","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2606.05647"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/CHATS-lab/coding-agent-safety-monitor","reach":null}],"summary":{"ran":6,"unverified":4},"by_repo_kind":{"found_in_text":{"samples":10,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f3fcaa199e3471a9","entry":"count_trajectory_events","repo":"CHATS-lab/coding-agent-safety-monitor","repo_kind":"found_in_text","path":"monitor/llm_monitor.py","file_url":"https://github.com/CHATS-lab/coding-agent-safety-monitor/blob/HEAD/monitor/llm_monitor.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f3fcaa199e3471a9"}},{"code_sha256_prefix":"ac02cd42ebd9a047","entry":"extract_json_from_response","repo":"CHATS-lab/coding-agent-safety-monitor","repo_kind":"found_in_text","path":"utils/io.py","file_url":"https://github.com/CHATS-lab/coding-agent-safety-monitor/blob/HEAD/utils/io.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ac02cd42ebd9a047"}},{"code_sha256_prefix":"6890cc4c38e729d8","entry":"get_bool_env","repo":"CHATS-lab/coding-agent-safety-monitor","repo_kind":"found_in_text","path":"utils/env.py","file_url":"https://github.com/CHATS-lab/coding-agent-safety-monitor/blob/HEAD/utils/env.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6890cc4c38e729d8"}},{"code_sha256_prefix":"d99b9efdc735877c","entry":"get_int_env","repo":"CHATS-lab/coding-agent-safety-monitor","repo_kind":"found_in_text","path":"utils/env.py","file_url":"https://github.com/CHATS-lab/coding-agent-safety-monitor/blob/HEAD/utils/env.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d99b9efdc735877c"}},{"code_sha256_prefix":"8add8881bedf4a28","entry":"read_jsonl","repo":"CHATS-lab/coding-agent-safety-monitor","repo_kind":"found_in_text","path":"utils/io.py","file_url":"https://github.com/CHATS-lab/coding-agent-safety-monitor/blob/HEAD/utils/io.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8add8881bedf4a28"}},{"code_sha256_prefix":"229e4957aa241a89","entry":"read_transcript","repo":"CHATS-lab/coding-agent-safety-monitor","repo_kind":"found_in_text","path":"utils/io.py","file_url":"https://github.com/CHATS-lab/coding-agent-safety-monitor/blob/HEAD/utils/io.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"229e4957aa241a89"}},{"code_sha256_prefix":"06861219c3db3670","entry":"format_trajectory","repo":"CHATS-lab/coding-agent-safety-monitor","repo_kind":"found_in_text","path":"monitor/llm_monitor.py","file_url":"https://github.com/CHATS-lab/coding-agent-safety-monitor/blob/HEAD/monitor/llm_monitor.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"06861219c3db3670"}},{"code_sha256_prefix":"fe6875812a147f8c","entry":"get_session_log_dir","repo":"CHATS-lab/coding-agent-safety-monitor","repo_kind":"found_in_text","path":"utils/logging.py","file_url":"https://github.com/CHATS-lab/coding-agent-safety-monitor/blob/HEAD/utils/logging.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe6875812a147f8c"}},{"code_sha256_prefix":"ae319c3778ecda71","entry":"setup_hook_logger","repo":"CHATS-lab/coding-agent-safety-monitor","repo_kind":"found_in_text","path":"utils/logging.py","file_url":"https://github.com/CHATS-lab/coding-agent-safety-monitor/blob/HEAD/utils/logging.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ae319c3778ecda71"}},{"code_sha256_prefix":"336aabf181e1fdd2","entry":"truncate_trajectory","repo":"CHATS-lab/coding-agent-safety-monitor","repo_kind":"found_in_text","path":"monitor/llm_monitor.py","file_url":"https://github.com/CHATS-lab/coding-agent-safety-monitor/blob/HEAD/monitor/llm_monitor.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"336aabf181e1fdd2"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}