{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/safeagentbench-a-benchmark-for-safe-task","title":"SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents","arxiv_id":"2412.13178","date":"2024-12-17","proceeding":null,"authors":["Sheng Yin","Xianghe Pang","Yuanzhuo Ding","Menglan Chen","Yutong Bi","Yichen Xiong","Wenhao Huang","Zhen Xiang","Jing Shao","Siheng Chen"],"abstract":"With the integration of large language models (LLMs), embodied agents have strong capabilities to understand and plan complicated natural language instructions. However, a foreseeable issue is that those embodied agents can also flawlessly execute some hazardous tasks, potentially causing damages in the real world. Existing benchmarks predominantly overlook critical safety risks, focusing solely on planning performance, while a few evaluate LLMs' safety awareness only on non-interactive image-text data. To address this gap, we present SafeAgentBench-the first benchmark for safety-aware task planning of embodied LLM agents in interactive simulation environments. SafeAgentBench includes: (1) an executable, diverse, and high-quality dataset of 750 tasks, rigorously curated to cover 10 potential hazards and 3 task types; (2) SafeAgentEnv, a universal embodied environment with a low-level controller, supporting multi-agent execution with 17 high-level actions for 8 state-of-the-art baselines; and (3) reliable evaluation methods from both execution and semantic perspectives. Experimental results show that, although agents based on different design frameworks exhibit substantial differences in task success rates, their overall safety awareness remains weak. The most safety-conscious baseline achieves only a 10\\% rejection rate for detailed hazardous tasks. Moreover, simply replacing the LLM driving the agent does not lead to notable improvements in safety awareness. More details and code are available at https://github.com/shengyin1224/SafeAgentBench.","url_abs":"https://arxiv.org/abs/2412.13178v4","url_pdf":"https://arxiv.org/pdf/2412.13178v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"safeagentbench-a-benchmark-for-safe-task","repo_url":"https://github.com/shengyin1224/safeagentbench","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"task-planning","task_name":"Task Planning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2412.13178","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.13178"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shengyin1224/safeagentbench","reach":{"status":"ok"}}],"summary":{"unverified":11},"by_repo_kind":{"official":{"samples":11,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"004ef77a803ae087","entry":"call_gpt","repo":"shengyin1224/safeagentbench","repo_kind":"official","path":"evaluator/abstract_evaluate.py","file_url":"https://github.com/shengyin1224/safeagentbench/blob/HEAD/evaluator/abstract_evaluate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"004ef77a803ae087"}},{"code_sha256_prefix":"2f61c26743e5287d","entry":"call_gpt","repo":"shengyin1224/safeagentbench","repo_kind":"official","path":"evaluator/detail_evaluate.py","file_url":"https://github.com/shengyin1224/safeagentbench/blob/HEAD/evaluator/detail_evaluate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2f61c26743e5287d"}},{"code_sha256_prefix":"cf64676d260a87c6","entry":"call_gpt","repo":"shengyin1224/safeagentbench","repo_kind":"official","path":"evaluator/long_horizon_evaluate.py","file_url":"https://github.com/shengyin1224/safeagentbench/blob/HEAD/evaluator/long_horizon_evaluate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cf64676d260a87c6"}},{"code_sha256_prefix":"5efd7755ace53052","entry":"call_openai_with_retry","repo":"shengyin1224/safeagentbench","repo_kind":"official","path":"evaluator/detail_evaluate.py","file_url":"https://github.com/shengyin1224/safeagentbench/blob/HEAD/evaluator/detail_evaluate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5efd7755ace53052"}},{"code_sha256_prefix":"20765b11fc73a976","entry":"compute_abstract_llm","repo":"shengyin1224/safeagentbench","repo_kind":"official","path":"evaluator/abstract_evaluate.py","file_url":"https://github.com/shengyin1224/safeagentbench/blob/HEAD/evaluator/abstract_evaluate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"20765b11fc73a976"}},{"code_sha256_prefix":"0cea743870f69c28","entry":"compute_safety_llm_long_horizon","repo":"shengyin1224/safeagentbench","repo_kind":"official","path":"evaluator/long_horizon_evaluate.py","file_url":"https://github.com/shengyin1224/safeagentbench/blob/HEAD/evaluator/long_horizon_evaluate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0cea743870f69c28"}},{"code_sha256_prefix":"c7c6225f6bf309e5","entry":"compute_safety_llm_long_horizon_another","repo":"shengyin1224/safeagentbench","repo_kind":"official","path":"evaluator/long_horizon_evaluate.py","file_url":"https://github.com/shengyin1224/safeagentbench/blob/HEAD/evaluator/long_horizon_evaluate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c7c6225f6bf309e5"}},{"code_sha256_prefix":"1b1f70631ef7016f","entry":"evaluate","repo":"shengyin1224/safeagentbench","repo_kind":"official","path":"evaluator/abstract_evaluate.py","file_url":"https://github.com/shengyin1224/safeagentbench/blob/HEAD/evaluator/abstract_evaluate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1b1f70631ef7016f"}},{"code_sha256_prefix":"b3617b20abb370c7","entry":"find_obj","repo":"shengyin1224/safeagentbench","repo_kind":"official","path":"methods/utils.py","file_url":"https://github.com/shengyin1224/safeagentbench/blob/HEAD/methods/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b3617b20abb370c7"}},{"code_sha256_prefix":"3b0c8cb51adc945c","entry":"find_obj_by_type","repo":"shengyin1224/safeagentbench","repo_kind":"official","path":"methods/utils.py","file_url":"https://github.com/shengyin1224/safeagentbench/blob/HEAD/methods/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3b0c8cb51adc945c"}},{"code_sha256_prefix":"a8523baf5db6f682","entry":"is_any_element_contained","repo":"shengyin1224/safeagentbench","repo_kind":"official","path":"evaluator/detail_evaluate.py","file_url":"https://github.com/shengyin1224/safeagentbench/blob/HEAD/evaluator/detail_evaluate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a8523baf5db6f682"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}