{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/transparency-and-explanation-in-deep","title":"Transparency and Explanation in Deep Reinforcement Learning Neural Networks","arxiv_id":"1809.06061","date":"2018-09-17","proceeding":null,"authors":["Rahul Iyer","Yuezhang Li","Huao Li","Michael Lewis","Ramitha Sundar","Katia Sycara"],"abstract":"Autonomous AI systems will be entering human society in the near future to\nprovide services and work alongside humans. For those systems to be accepted\nand trusted, the users should be able to understand the reasoning process of\nthe system, i.e. the system should be transparent. System transparency enables\nhumans to form coherent explanations of the system's decisions and actions.\nTransparency is important not only for user trust, but also for software\ndebugging and certification. In recent years, Deep Neural Networks have made\ngreat advances in multiple application areas. However, deep neural networks are\nopaque. In this paper, we report on work in transparency in Deep Reinforcement\nLearning Networks (DRLN). Such networks have been extremely successful in\naccurately learning action control in image input domains, such as Atari games.\nIn this paper, we propose a novel and general method that (a) incorporates\nexplicit object recognition processing into deep reinforcement learning models,\n(b) forms the basis for the development of \"object saliency maps\", to provide\nvisualization of internal states of DRLNs, thus enabling the formation of\nexplanations and (c) can be incorporated in any existing deep reinforcement\nlearning framework. We present computational results and human experiments to\nevaluate our approach.","url_abs":"http://arxiv.org/abs/1809.06061v1","url_pdf":"http://arxiv.org/pdf/1809.06061v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"transparency-and-explanation-in-deep","repo_url":"https://github.com/KDL-umass/saliency_maps","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"object-recognition","task_name":"Object Recognition"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1809.06061","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}