{"url":"/method/experience-replay","slug":"experience-replay","name":"Experience Replay","full_name":"Experience Replay","full_name_withheld":false,"description_markdown":"**Experience Replay** is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\\_{t} = \\left(s\\_{t}, a\\_{t}, r\\_{t}, s\\_{t+1}\\right)$ in a data-set $D = e\\_{1}, \\cdots, e\\_{N}$ , pooled over many episodes into a replay memory. We then usually sample the memory randomly for a minibatch of experience, and use this to learn off-policy, as with Deep Q-Networks. This tackles the problem of autocorrelation leading to unstable training, by making the problem more like a supervised learning problem.\r\n\r\nImage Credit: [Hands-On Reinforcement Learning with Python, Sudharsan Ravichandiran](https://subscription.packtpub.com/book/big_data_and_business_intelligence/9781788836524)","description_state":"present","introduced_year":1993,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":null,"title":null,"url_on_a_paper_host":false},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Reinforcement Learning","area_id":"reinforcement-learning","collection":"Replay Memory","url":"/methods/category/replay-memory","pwc_aliases":[]}],"n_papers_tagged":865,"archive_num_papers":865,"papers_newest_first":[{"paper":null,"title":"Turning Sand to Gold: Recycling Data to Bridge On-Policy and Off-Policy Learning via Causal Bound","date":"2025-07-15","arxiv_id":"2507.11269","n_code_links":0,"syntology":null},{"paper":"/paper/deep-reinforcement-learning-with-gradient-1","title":"Deep Reinforcement Learning with Gradient Eligibility Traces","date":"2025-07-12","arxiv_id":"2507.09087","n_code_links":1,"syntology":null},{"paper":null,"title":"Multi-Objective Reinforcement Learning for Cognitive Radar Resource Management","date":"2025-06-25","arxiv_id":"2506.20853","n_code_links":0,"syntology":null},{"paper":"/paper/cawr-corruption-averse-advantage-weighted","title":"CAWR: Corruption-Averse Advantage-Weighted Regression for Robust Policy Optimization","date":"2025-06-18","arxiv_id":"2506.15654","n_code_links":1,"syntology":null},{"paper":null,"title":"Reliable Critics: Monotonic Improvement and Convergence Guarantees for Reinforcement Learning","date":"2025-06-08","arxiv_id":"2506.07134","n_code_links":0,"syntology":null},{"paper":null,"title":"Contextual Experience Replay for Self-Improvement of Language Agents","date":"2025-06-07","arxiv_id":"2506.06698","n_code_links":0,"syntology":null},{"paper":"/paper/gradual-transition-from-bellman-optimality","title":"Gradual Transition from Bellman Optimality Operator to Bellman Operator in Online Reinforcement Learning","date":"2025-06-06","arxiv_id":"2506.05968","n_code_links":1,"syntology":{"ran":1,"of":4,"unverified":3,"pointer_only":0}},{"paper":null,"title":"A Novel Deep Reinforcement Learning Method for Computation Offloading in Multi-User Mobile Edge Computing with Decentralization","date":"2025-06-03","arxiv_id":"2506.02458","n_code_links":0,"syntology":null},{"paper":null,"title":"FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Control","date":"2025-05-28","arxiv_id":"2505.22642","n_code_links":0,"syntology":null},{"paper":null,"title":"Analyzing Mitigation Strategies for Catastrophic Forgetting in End-to-End Training of Spoken Language Models","date":"2025-05-23","arxiv_id":"2505.17496","n_code_links":0,"syntology":null},{"paper":null,"title":"LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models","date":"2025-05-21","arxiv_id":"2505.15293","n_code_links":0,"syntology":null},{"paper":"/paper/lifelongagentbench-evaluating-llm-agents-as","title":"LifelongAgentBench: Evaluating LLM Agents as Lifelong Learners","date":"2025-05-17","arxiv_id":"2505.11942","n_code_links":1,"syntology":null},{"paper":null,"title":"Unveiling the Black Box: A Multi-Layer Framework for Explaining Reinforcement Learning-Based Cyber Agents","date":"2025-05-16","arxiv_id":"2505.11708","n_code_links":0,"syntology":null},{"paper":null,"title":"Electric Bus Charging Schedules Relying on Real Data-Driven Targets Based on Hierarchical Deep Reinforcement Learning","date":"2025-05-15","arxiv_id":"2505.10262","n_code_links":0,"syntology":null},{"paper":null,"title":"Deep reinforcement learning-based longitudinal control strategy for automated vehicles at signalised intersections","date":"2025-05-13","arxiv_id":"2505.08896","n_code_links":0,"syntology":null},{"paper":null,"title":"GradMix: Gradient-based Selective Mixup for Robust Data Augmentation in Class-Incremental Learning","date":"2025-05-13","arxiv_id":"2505.08528","n_code_links":0,"syntology":null},{"paper":null,"title":"Monte Carlo Beam Search for Actor-Critic Reinforcement Learning in Continuous Control","date":"2025-05-13","arxiv_id":"2505.09029","n_code_links":0,"syntology":null},{"paper":"/paper/online-learning-based-adaptive-beam-switching","title":"Online Learning-based Adaptive Beam Switching for 6G Networks: Enhancing Efficiency and Resilience","date":"2025-05-12","arxiv_id":"2505.08032","n_code_links":1,"syntology":null},{"paper":null,"title":"Energy Efficient RSMA-Based LEO Satellite Communications Assisted by UAV-Mounted BD-Active RIS: A DRL Approach","date":"2025-05-07","arxiv_id":"2505.04148","n_code_links":0,"syntology":null},{"paper":null,"title":"A Goal-Oriented Reinforcement Learning-Based Path Planning Algorithm for Modular Self-Reconfigurable Satellites","date":"2025-05-04","arxiv_id":"2505.01966","n_code_links":0,"syntology":null},{"paper":null,"title":"Approximation to Deep Q-Network by Stochastic Delay Differential Equations","date":"2025-05-01","arxiv_id":"2505.00382","n_code_links":0,"syntology":null},{"paper":null,"title":"AlphaGrad: Non-Linear Gradient Normalization Optimizer","date":"2025-04-22","arxiv_id":"2504.16020","n_code_links":0,"syntology":null},{"paper":null,"title":"Next-Future: Sample-Efficient Policy Learning for Robotic-Arm Tasks","date":"2025-04-15","arxiv_id":"2504.11247","n_code_links":0,"syntology":null},{"paper":null,"title":"Moderate Actor-Critic Methods: Controlling Overestimation Bias via Expectile Loss","date":"2025-04-14","arxiv_id":"2504.09929","n_code_links":0,"syntology":null},{"paper":null,"title":"Graph Based Deep Reinforcement Learning Aided by Transformers for Multi-Agent Cooperation","date":"2025-04-11","arxiv_id":"2504.08195","n_code_links":0,"syntology":null},{"paper":"/paper/proxy-anchor-and-evt-driven-continual","title":"Proxy-Anchor and EVT-Driven Continual Learning Method for Generalized Category Discovery","date":"2025-04-11","arxiv_id":"2504.08550","n_code_links":1,"syntology":null},{"paper":null,"title":"Neuron-level Balance between Stability and Plasticity in Deep Reinforcement Learning","date":"2025-04-09","arxiv_id":"2504.08000","n_code_links":0,"syntology":null},{"paper":null,"title":"Think Small, Act Big: Primitive Prompt Learning for Lifelong Robot Manipulation","date":"2025-04-01","arxiv_id":"2504.00420","n_code_links":0,"syntology":null},{"paper":null,"title":"Dynamic Operating System Scheduling Using Double DQN: A Reinforcement Learning Approach to Task Optimization","date":"2025-03-31","arxiv_id":"2503.23659","n_code_links":0,"syntology":null},{"paper":"/paper/pareto-continual-learning-preference","title":"Pareto Continual Learning: Preference-Conditioned Learning and Adaption for Dynamic Stability-Plasticity Trade-off","date":"2025-03-30","arxiv_id":"2503.23390","n_code_links":1,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":440},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":375},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":371},{"task":"/task/deep-reinforcement-learning","name":"Deep Reinforcement Learning","papers":296},{"task":"/task/continuous-control","name":"Continuous Control","papers":99},{"task":"/task/q-learning","name":"Q-Learning","papers":94},{"task":"/task/continuous-control","name":"continuous-control","papers":93},{"task":"/task/continual-learning","name":"Continual Learning","papers":90},{"task":"/task/mujoco","name":"MuJoCo","papers":54},{"task":"/task/decision-making","name":"Decision Making","papers":49},{"task":"/task/openai-gym","name":"OpenAI Gym","papers":43},{"task":"/task/multi-agent-reinforcement-learning","name":"Multi-agent Reinforcement Learning","papers":42},{"task":"/task/atari-games","name":"Atari Games","papers":34},{"task":"/task/management","name":"Management","papers":25},{"task":"/task/autonomous-driving","name":"Autonomous Driving","papers":22},{"task":"/task/imitation-learning","name":"Imitation Learning","papers":21},{"task":"/task/incremental-learning","name":"Incremental Learning","papers":21},{"task":"/task/scheduling","name":"Scheduling","papers":20},{"task":"/task/diversity","name":"Diversity","papers":17},{"task":"/task/multi-goal-reinforcement-learning","name":"Multi-Goal Reinforcement Learning","papers":17}],"tasks_shown":20,"n_tasks":289,"usage_by_year":[{"year":"2013","papers":1},{"year":"2015","papers":3},{"year":"2016","papers":5},{"year":"2017","papers":17},{"year":"2018","papers":57},{"year":"2019","papers":86},{"year":"2020","papers":129},{"year":"2021","papers":145},{"year":"2022","papers":113},{"year":"2023","papers":134},{"year":"2024","papers":117},{"year":"2025","papers":58}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/experience-replay"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}