{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/evil-geniuses-delving-into-the-safety-of-llm","title":"Evil Geniuses: Delving into the Safety of LLM-based Agents","arxiv_id":"2311.11855","date":"2023-11-20","proceeding":null,"authors":["Yu Tian","Xiao Yang","Jingyuan Zhang","Yinpeng Dong","Hang Su"],"abstract":"Rapid advancements in large language models (LLMs) have revitalized in LLM-based agents, exhibiting impressive human-like behaviors and cooperative capabilities in various scenarios. However, these agents also bring some exclusive risks, stemming from the complexity of interaction environments and the usability of tools. This paper delves into the safety of LLM-based agents from three perspectives: agent quantity, role definition, and attack level. Specifically, we initially propose to employ a template-based attack strategy on LLM-based agents to find the influence of agent quantity. In addition, to address interaction environment and role specificity issues, we introduce Evil Geniuses (EG), an effective attack method that autonomously generates prompts related to the original role to examine the impact across various role definitions and attack levels. EG leverages Red-Blue exercises, significantly improving the generated prompt aggressiveness and similarity to original roles. Our evaluations on CAMEL, Metagpt and ChatDev based on GPT-3.5 and GPT-4, demonstrate high success rates. Extensive evaluation and discussion reveal that these agents are less robust, prone to more harmful behaviors, and capable of generating stealthier content than LLMs, highlighting significant safety challenges and guiding future research. Our code is available at https://github.com/T1aNS1R/Evil-Geniuses.","url_abs":"https://arxiv.org/abs/2311.11855v2","url_pdf":"https://arxiv.org/pdf/2311.11855v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"evil-geniuses-delving-into-the-safety-of-llm","repo_url":"https://github.com/t1ans1r/evil-geniuses","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"specificity","task_name":"Specificity"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"cosine-annealing","method_name":"Cosine Annealing"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-3","method_name":"GPT-3"},{"method_slug":"gpt-4","method_name":"GPT-4"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-cosine-annealing","method_name":"Linear Warmup With Cosine Annealing"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2311.11855","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.11855"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/t1ans1r/evil-geniuses","reach":{"status":"ok"}}],"summary":{"ran":3,"unverified":2},"by_repo_kind":{"official":{"samples":5,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"72fd8c6bbf231255","entry":"load_roles","repo":"t1ans1r/evil-geniuses","repo_kind":"official","path":"Evil_Geniuses/apps/agents/agents.py","file_url":"https://github.com/t1ans1r/evil-geniuses/blob/HEAD/Evil_Geniuses/apps/agents/agents.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"72fd8c6bbf231255"}},{"code_sha256_prefix":"e9a58fed93c4a746","entry":"split_markdown_code","repo":"t1ans1r/evil-geniuses","repo_kind":"official","path":"Evil_Geniuses/apps/agents/text_utils.py","file_url":"https://github.com/t1ans1r/evil-geniuses/blob/HEAD/Evil_Geniuses/apps/agents/text_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e9a58fed93c4a746"}},{"code_sha256_prefix":"a540d84d0966eea3","entry":"update_license_in_file","repo":"t1ans1r/evil-geniuses","repo_kind":"official","path":"Evil_Geniuses/licenses/update_license.py","file_url":"https://github.com/t1ans1r/evil-geniuses/blob/HEAD/Evil_Geniuses/licenses/update_license.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a540d84d0966eea3"}},{"code_sha256_prefix":"b82e216b9c242c24","entry":"find_license_end_line","repo":"t1ans1r/evil-geniuses","repo_kind":"official","path":"Evil_Geniuses/licenses/update_license.py","file_url":"https://github.com/t1ans1r/evil-geniuses/blob/HEAD/Evil_Geniuses/licenses/update_license.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b82e216b9c242c24"}},{"code_sha256_prefix":"b282d1daf310745e","entry":"fine_license_start_line","repo":"t1ans1r/evil-geniuses","repo_kind":"official","path":"Evil_Geniuses/licenses/update_license.py","file_url":"https://github.com/t1ans1r/evil-geniuses/blob/HEAD/Evil_Geniuses/licenses/update_license.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b282d1daf310745e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}