{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/using-state-predictions-for-value","title":"Using State Predictions for Value Regularization in Curiosity Driven Deep Reinforcement Learning","arxiv_id":"1810.00361","date":"2018-09-30","proceeding":null,"authors":["Gino Brunner","Manuel Fritsche","Oliver Richter","Roger Wattenhofer"],"abstract":"Learning in sparse reward settings remains a challenge in Reinforcement\nLearning, which is often addressed by using intrinsic rewards. One promising\nstrategy is inspired by human curiosity, requiring the agent to learn to\npredict the future. In this paper a curiosity-driven agent is extended to use\nthese predictions directly for training. To achieve this, the agent predicts\nthe value function of the next state at any point in time. Subsequently, the\nconsistency of this prediction with the current value function is measured,\nwhich is then used as a regularization term in the loss function of the\nalgorithm. Experiments were made on grid-world environments as well as on a 3D\nnavigation task, both with sparse rewards. In the first case the extended agent\nis able to learn significantly faster than the baselines.","url_abs":"http://arxiv.org/abs/1810.00361v1","url_pdf":"http://arxiv.org/pdf/1810.00361v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"using-state-predictions-for-value","repo_url":"https://github.com/ManuelFritsche/vpc","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}