{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/from-allies-to-adversaries-manipulating-llm","title":"From Allies to Adversaries: Manipulating LLM Tool-Calling through Adversarial Injection","arxiv_id":"2412.10198","date":"2024-12-13","proceeding":null,"authors":["Haowei Wang","Rupeng Zhang","Junjie Wang","Mingyang Li","Yuekai Huang","Dandan Wang","Qing Wang"],"abstract":"Tool-calling has changed Large Language Model (LLM) applications by integrating external tools, significantly enhancing their functionality across diverse tasks. However, this integration also introduces new security vulnerabilities, particularly in the tool scheduling mechanisms of LLM, which have not been extensively studied. To fill this gap, we present ToolCommander, a novel framework designed to exploit vulnerabilities in LLM tool-calling systems through adversarial tool injection. Our framework employs a well-designed two-stage attack strategy. Firstly, it injects malicious tools to collect user queries, then dynamically updates the injected tools based on the stolen information to enhance subsequent attacks. These stages enable ToolCommander to execute privacy theft, launch denial-of-service attacks, and even manipulate business competition by triggering unscheduled tool-calling. Notably, the ASR reaches 91.67% for privacy theft and hits 100% for denial-of-service and unscheduled tool calling in certain cases. Our work demonstrates that these vulnerabilities can lead to severe consequences beyond simple misuse of tool-calling systems, underscoring the urgent need for robust defensive strategies to secure LLM Tool-calling systems.","url_abs":"https://arxiv.org/abs/2412.10198v2","url_pdf":"https://arxiv.org/pdf/2412.10198v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"from-allies-to-adversaries-manipulating-llm","repo_url":"https://github.com/anonymous-lgtm/toolcommander","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"large-language-model","task_name":"Large Language Model"},{"task_slug":"scheduling","task_name":"Scheduling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2412.10198","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.10198"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/anonymous-lgtm/toolcommander","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"ran":1,"unverified":7},"by_repo_kind":{"official":{"samples":9,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e6d1b09cb3624d3f","entry":"standardize","repo":"anonymous-lgtm/toolcommander","repo_kind":"official","path":"toolbench.py","file_url":"https://github.com/anonymous-lgtm/toolcommander/blob/HEAD/toolbench.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e6d1b09cb3624d3f"}},{"code_sha256_prefix":"04eed77ef8a3068b","entry":"standardize_category","repo":"anonymous-lgtm/toolcommander","repo_kind":"official","path":"toolbench.py","file_url":"https://github.com/anonymous-lgtm/toolcommander/blob/HEAD/toolbench.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"04eed77ef8a3068b"}},{"code_sha256_prefix":"63c9d2322eee62a3","entry":"clear_cache_decorator","repo":"anonymous-lgtm/toolcommander","repo_kind":"official","path":"utils.py","file_url":"https://github.com/anonymous-lgtm/toolcommander/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"63c9d2322eee62a3"}},{"code_sha256_prefix":"26a671465d6d3f16","entry":"data_dict_to_idxs","repo":"anonymous-lgtm/toolcommander","repo_kind":"official","path":"evaluate.py","file_url":"https://github.com/anonymous-lgtm/toolcommander/blob/HEAD/evaluate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"26a671465d6d3f16"}},{"code_sha256_prefix":"7b2d4c6e3c00ff7e","entry":"generate_batch_oai","repo":"anonymous-lgtm/toolcommander","repo_kind":"official","path":"llm.py","file_url":"https://github.com/anonymous-lgtm/toolcommander/blob/HEAD/llm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7b2d4c6e3c00ff7e"}},{"code_sha256_prefix":"09dbda0cade0d1b2","entry":"get_eat_other_response","repo":"anonymous-lgtm/toolcommander","repo_kind":"official","path":"utils.py","file_url":"https://github.com/anonymous-lgtm/toolcommander/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"09dbda0cade0d1b2"}},{"code_sha256_prefix":"78c303cdd20b9bc3","entry":"load_tool_json","repo":"anonymous-lgtm/toolcommander","repo_kind":"official","path":"toolbench.py","file_url":"https://github.com/anonymous-lgtm/toolcommander/blob/HEAD/toolbench.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"78c303cdd20b9bc3"}},{"code_sha256_prefix":"e639e9c332c5469a","entry":"loss_calculator_wrapper","repo":"anonymous-lgtm/toolcommander","repo_kind":"official","path":"utils.py","file_url":"https://github.com/anonymous-lgtm/toolcommander/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e639e9c332c5469a"}},{"code_sha256_prefix":"01aca5c155fa5652","entry":"token_gradients","repo":"anonymous-lgtm/toolcommander","repo_kind":"official","path":"mcg.py","file_url":"https://github.com/anonymous-lgtm/toolcommander/blob/HEAD/mcg.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"01aca5c155fa5652"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}