{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/advances-in-experience-replay","title":"Advances in Experience Replay","arxiv_id":"1805.05536","date":"2018-05-15","proceeding":null,"authors":["Tracy Wan","Neil Xu"],"abstract":"This project combines recent advances in experience replay techniques,\nnamely, Combined Experience Replay (CER), Prioritized Experience Replay (PER),\nand Hindsight Experience Replay (HER). We show the results of combinations of\nthese techniques with DDPG and DQN methods. CER always adds the most recent\nexperience to the batch. PER chooses which experiences should be replayed based\non how beneficial they will be towards learning. HER learns from failure by\nsubstituting the desired goal with the achieved goal and recomputing the reward\nfunction. The effectiveness of combinations of these experience replay\ntechniques is tested in a variety of OpenAI gym environments.","url_abs":"http://arxiv.org/abs/1805.05536v1","url_pdf":"http://arxiv.org/pdf/1805.05536v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"advances-in-experience-replay","repo_url":"https://github.com/himat/CHAPtER","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"openai-gym","task_name":"OpenAI Gym"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"ddpg","method_name":"DDPG"},{"method_slug":"dqn","method_name":"DQN"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"experience-replay","method_name":"Experience Replay"},{"method_slug":"prioritized-experience-replay","method_name":"Prioritized Experience Replay"},{"method_slug":"q-learning","method_name":"Q-Learning"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}