{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/local-and-global-explanations-of-agent","title":"Local and Global Explanations of Agent Behavior: Integrating Strategy Summaries with Saliency Maps","arxiv_id":"2005.08874","date":"2020-05-18","proceeding":null,"authors":["Tobias Huber","Katharina Weitz","Elisabeth André","Ofra Amir"],"abstract":"With advances in reinforcement learning (RL), agents are now being developed in high-stakes application domains such as healthcare and transportation. Explaining the behavior of these agents is challenging, as the environments in which they act have large state spaces, and their decision-making can be affected by delayed rewards, making it difficult to analyze their behavior. To address this problem, several approaches have been developed. Some approaches attempt to convey the $\\textit{global}$ behavior of the agent, describing the actions it takes in different states. Other approaches devised $\\textit{local}$ explanations which provide information regarding the agent's decision-making in a particular state. In this paper, we combine global and local explanation methods, and evaluate their joint and separate contributions, providing (to the best of our knowledge) the first user study of combined local and global explanations for RL agents. Specifically, we augment strategy summaries that extract important trajectories of states from simulations of the agent with saliency maps which show what information the agent attends to. Our results show that the choice of what states to include in the summary (global information) strongly affects people's understanding of agents: participants shown summaries that included important states significantly outperformed participants who were presented with agent behavior in a randomly set of chosen world-states. We find mixed results with respect to augmenting demonstrations with saliency maps (local information), as the addition of saliency maps did not significantly improve performance in most cases. However, we do find some evidence that saliency maps can help users better understand what information the agent relies on in its decision making, suggesting avenues for future work that can further improve explanations of RL agents.","url_abs":"https://arxiv.org/abs/2005.08874v3","url_pdf":"https://arxiv.org/pdf/2005.08874v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"local-and-global-explanations-of-agent","repo_url":"https://github.com/HuTobias/HIGHLIGHTS-LRP","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"feature-importance","task_name":"Feature Importance"},{"task_slug":"general-reinforcement-learning","task_name":"General Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dqn","method_name":"DQN"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"q-learning","method_name":"Q-Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2005.08874","atlas_url":"https://app.syntology.ai/?focus=2005.08874","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2005.08874"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/HuTobias/HIGHLIGHTS-LRP","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":15},"by_repo_kind":{"official":{"samples":15,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6bf777cae5aa1bbe","entry":"add_saliency_to_image","repo":"HuTobias/HIGHLIGHTS-LRP","repo_kind":"official","path":"image_utils.py","file_url":"https://github.com/HuTobias/HIGHLIGHTS-LRP/blob/HEAD/image_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6bf777cae5aa1bbe"}},{"code_sha256_prefix":"23b5e027f63446a1","entry":"boot_matrix","repo":"HuTobias/HIGHLIGHTS-LRP","repo_kind":"official","path":"Survey_results/evaluation.py","file_url":"https://github.com/HuTobias/HIGHLIGHTS-LRP/blob/HEAD/Survey_results/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"23b5e027f63446a1"}},{"code_sha256_prefix":"15507b00dcf9ed2e","entry":"bootstrap_mean","repo":"HuTobias/HIGHLIGHTS-LRP","repo_kind":"official","path":"Survey_results/evaluation.py","file_url":"https://github.com/HuTobias/HIGHLIGHTS-LRP/blob/HEAD/Survey_results/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"15507b00dcf9ed2e"}},{"code_sha256_prefix":"ec5245b230cd1b7f","entry":"check_group","repo":"HuTobias/HIGHLIGHTS-LRP","repo_kind":"official","path":"Survey_results/evaluate_text.py","file_url":"https://github.com/HuTobias/HIGHLIGHTS-LRP/blob/HEAD/Survey_results/evaluate_text.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ec5245b230cd1b7f"}},{"code_sha256_prefix":"3ee689932999e7d2","entry":"create_edge_image","repo":"HuTobias/HIGHLIGHTS-LRP","repo_kind":"official","path":"image_utils.py","file_url":"https://github.com/HuTobias/HIGHLIGHTS-LRP/blob/HEAD/image_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3ee689932999e7d2"}},{"code_sha256_prefix":"456a33d3018f4f34","entry":"find_similar_state_in_summary","repo":"HuTobias/HIGHLIGHTS-LRP","repo_kind":"official","path":"highlights_state_selection.py","file_url":"https://github.com/HuTobias/HIGHLIGHTS-LRP/blob/HEAD/highlights_state_selection.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"456a33d3018f4f34"}},{"code_sha256_prefix":"16a1e82ed27c1ffa","entry":"get_feature_vector","repo":"HuTobias/HIGHLIGHTS-LRP","repo_kind":"official","path":"stream_generator.py","file_url":"https://github.com/HuTobias/HIGHLIGHTS-LRP/blob/HEAD/stream_generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"16a1e82ed27c1ffa"}},{"code_sha256_prefix":"68854d095267927a","entry":"get_grouped_df","repo":"HuTobias/HIGHLIGHTS-LRP","repo_kind":"official","path":"Survey_results/evaluate_text.py","file_url":"https://github.com/HuTobias/HIGHLIGHTS-LRP/blob/HEAD/Survey_results/evaluate_text.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"68854d095267927a"}},{"code_sha256_prefix":"d684cfecadd9fc63","entry":"highlights","repo":"HuTobias/HIGHLIGHTS-LRP","repo_kind":"official","path":"highlights_state_selection.py","file_url":"https://github.com/HuTobias/HIGHLIGHTS-LRP/blob/HEAD/highlights_state_selection.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d684cfecadd9fc63"}},{"code_sha256_prefix":"d95a6badeecdce28","entry":"interpolate","repo":"HuTobias/HIGHLIGHTS-LRP","repo_kind":"official","path":"overlay_stream.py","file_url":"https://github.com/HuTobias/HIGHLIGHTS-LRP/blob/HEAD/overlay_stream.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d95a6badeecdce28"}},{"code_sha256_prefix":"41749c3cf919738c","entry":"load_Model_with_trained_variables","repo":"HuTobias/HIGHLIGHTS-LRP","repo_kind":"official","path":"Tensorflow_to_Keras.py","file_url":"https://github.com/HuTobias/HIGHLIGHTS-LRP/blob/HEAD/Tensorflow_to_Keras.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"41749c3cf919738c"}},{"code_sha256_prefix":"bedbfacf131fc2a1","entry":"output_saliency_map","repo":"HuTobias/HIGHLIGHTS-LRP","repo_kind":"official","path":"image_utils.py","file_url":"https://github.com/HuTobias/HIGHLIGHTS-LRP/blob/HEAD/image_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bedbfacf131fc2a1"}},{"code_sha256_prefix":"3a44724f5fec1a11","entry":"random_state_selection","repo":"HuTobias/HIGHLIGHTS-LRP","repo_kind":"official","path":"highlights_state_selection.py","file_url":"https://github.com/HuTobias/HIGHLIGHTS-LRP/blob/HEAD/highlights_state_selection.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3a44724f5fec1a11"}},{"code_sha256_prefix":"e0d84f47df26823f","entry":"rank_biserial_effect_size","repo":"HuTobias/HIGHLIGHTS-LRP","repo_kind":"official","path":"Survey_results/evaluation.py","file_url":"https://github.com/HuTobias/HIGHLIGHTS-LRP/blob/HEAD/Survey_results/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e0d84f47df26823f"}},{"code_sha256_prefix":"eeb60e3bda5f2978","entry":"read_text_data","repo":"HuTobias/HIGHLIGHTS-LRP","repo_kind":"official","path":"Survey_results/evaluate_text.py","file_url":"https://github.com/HuTobias/HIGHLIGHTS-LRP/blob/HEAD/Survey_results/evaluate_text.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eeb60e3bda5f2978"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}