{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/psysafe-a-comprehensive-framework-for","title":"PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety","arxiv_id":"2401.11880","date":"2024-01-22","proceeding":null,"authors":["Zaibin Zhang","Yongting Zhang","Lijun Li","Hongzhi Gao","Lijun Wang","Huchuan Lu","Feng Zhao","Yu Qiao","Jing Shao"],"abstract":"Multi-agent systems, when enhanced with Large Language Models (LLMs), exhibit profound capabilities in collective intelligence. However, the potential misuse of this intelligence for malicious purposes presents significant risks. To date, comprehensive research on the safety issues associated with multi-agent systems remains limited. In this paper, we explore these concerns through the innovative lens of agent psychology, revealing that the dark psychological states of agents constitute a significant threat to safety. To tackle these concerns, we propose a comprehensive framework (PsySafe) grounded in agent psychology, focusing on three key areas: firstly, identifying how dark personality traits in agents can lead to risky behaviors; secondly, evaluating the safety of multi-agent systems from the psychological and behavioral perspectives, and thirdly, devising effective strategies to mitigate these risks. Our experiments reveal several intriguing phenomena, such as the collective dangerous behaviors among agents, agents' self-reflection when engaging in dangerous behavior, and the correlation between agents' psychological assessments and dangerous behaviors. We anticipate that our framework and observations will provide valuable insights for further research into the safety of multi-agent systems. We will make our data and code publicly accessible at https://github.com/AI4Good24/PsySafe.","url_abs":"https://arxiv.org/abs/2401.11880v3","url_pdf":"https://arxiv.org/pdf/2401.11880v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"psysafe-a-comprehensive-framework-for","repo_url":"https://github.com/ai4good24/psysafe","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"psysafe-a-comprehensive-framework-for","repo_url":"https://github.com/guozhenyuan/phimm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2401.11880","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.11880"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/guozhenyuan/phimm","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ai4good24/psysafe","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/AI4Good24/PsySafe","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1,"ran":8},"by_repo_kind":{"official":{"samples":9,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"7b33aa6ea6a4f06c","entry":"build_human_attack","repo":"AI4Good24/PsySafe","repo_kind":"official","path":"build_prompt.py","file_url":"https://github.com/AI4Good24/PsySafe/blob/HEAD/build_prompt.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7b33aa6ea6a4f06c"}},{"code_sha256_prefix":"888b3bd659f238b8","entry":"compute_agent_psy","repo":"ai4good24/psysafe","repo_kind":"official","path":"result_extract.py","file_url":"https://github.com/ai4good24/psysafe/blob/HEAD/result_extract.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"888b3bd659f238b8"}},{"code_sha256_prefix":"6992542cb57eb327","entry":"convert_results","repo":"ai4good24/psysafe","repo_kind":"official","path":"round_extract.py","file_url":"https://github.com/ai4good24/psysafe/blob/HEAD/round_extract.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6992542cb57eb327"}},{"code_sha256_prefix":"a515d69eb855d6a8","entry":"count_occurrences","repo":"ai4good24/psysafe","repo_kind":"official","path":"result_extract.py","file_url":"https://github.com/ai4good24/psysafe/blob/HEAD/result_extract.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a515d69eb855d6a8"}},{"code_sha256_prefix":"19fabd7f1dd76938","entry":"eval_agents","repo":"ai4good24/psysafe","repo_kind":"official","path":"result_extract.py","file_url":"https://github.com/ai4good24/psysafe/blob/HEAD/result_extract.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"19fabd7f1dd76938"}},{"code_sha256_prefix":"2173338dd76bb926","entry":"gen_moral_prompt","repo":"ai4good24/psysafe","repo_kind":"official","path":"start.py","file_url":"https://github.com/ai4good24/psysafe/blob/HEAD/start.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2173338dd76bb926"}},{"code_sha256_prefix":"7357e9255fe51a71","entry":"get_questionnaire","repo":"ai4good24/psysafe","repo_kind":"official","path":"round_extract.py","file_url":"https://github.com/ai4good24/psysafe/blob/HEAD/round_extract.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7357e9255fe51a71"}},{"code_sha256_prefix":"5f6710c70514745e","entry":"load_json","repo":"ai4good24/psysafe","repo_kind":"official","path":"round_extract.py","file_url":"https://github.com/ai4good24/psysafe/blob/HEAD/round_extract.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5f6710c70514745e"}},{"code_sha256_prefix":"b3f1b84719c972bd","entry":"task_specify","repo":"ai4good24/psysafe","repo_kind":"official","path":"start.py","file_url":"https://github.com/ai4good24/psysafe/blob/HEAD/start.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b3f1b84719c972bd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}