{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/empowerment-driven-exploration-using-mutual","title":"Empowerment-driven Exploration using Mutual Information Estimation","arxiv_id":"1810.05533","date":"2018-10-11","proceeding":null,"authors":["Navneet Madhu Kumar"],"abstract":"Exploration is a difficult challenge in reinforcement learning and is of\nprime importance in sparse reward environments. However, many of the state of\nthe art deep reinforcement learning algorithms, that rely on epsilon-greedy,\nfail on these environments. In such cases, empowerment can serve as an\nintrinsic reward signal to enable the agent to maximize the influence it has\nover the near future. We formulate empowerment as the channel capacity between\nstates and actions and is calculated by estimating the mutual information\nbetween the actions and the following states. The mutual information is\nestimated using Mutual Information Neural Estimator and a forward dynamics\nmodel. We demonstrate that an empowerment driven agent is able to improve\nsignificantly the score of a baseline DQN agent on the game of Montezuma's\nRevenge.","url_abs":"http://arxiv.org/abs/1810.05533v1","url_pdf":"http://arxiv.org/pdf/1810.05533v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"empowerment-driven-exploration-using-mutual","repo_url":"https://github.com/navneet-nmk/pytorch-rl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"montezumas-revenge","task_name":"Montezuma's Revenge"},{"task_slug":"mutual-information-estimation","task_name":"Mutual Information Estimation"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dqn","method_name":"DQN"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"q-learning","method_name":"Q-Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}