{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/automated-curriculum-learning-by-rewarding","title":"Automated Curriculum Learning by Rewarding Temporally Rare Events","arxiv_id":"1803.07131","date":"2018-03-19","proceeding":null,"authors":["Niels Justesen","Sebastian Risi"],"abstract":"Reward shaping allows reinforcement learning (RL) agents to accelerate\nlearning by receiving additional reward signals. However, these signals can be\ndifficult to design manually, especially for complex RL tasks. We propose a\nsimple and general approach that determines the reward of pre-defined events by\ntheir rarity alone. Here events become less rewarding as they are experienced\nmore often, which encourages the agent to continually explore new types of\nevents as it learns. The adaptiveness of this reward function results in a form\nof automated curriculum learning that does not have to be specified by the\nexperimenter. We demonstrate that this \\emph{Rarity of Events} (RoE) approach\nenables the agent to succeed in challenging VizDoom scenarios without access to\nthe extrinsic reward from the environment. Furthermore, the results demonstrate\nthat RoE learns a more versatile policy that adapts well to critical changes in\nthe environment. Rewarding events based on their rarity could help in many\nunsolved RL environments that are characterized by sparse extrinsic rewards but\na plethora of known event types.","url_abs":"http://arxiv.org/abs/1803.07131v2","url_pdf":"http://arxiv.org/pdf/1803.07131v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"automated-curriculum-learning-by-rewarding","repo_url":"https://github.com/lasseuth1/blood_bowl2","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1803.07131","atlas_url":"https://app.syntology.ai/?focus=1803.07131","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}