{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/catastrophic-interference-in-reinforcement","title":"Catastrophic Interference in Reinforcement Learning: A Solution Based on Context Division and Knowledge Distillation","arxiv_id":"2109.00525","date":"2021-09-01","proceeding":null,"authors":["Tiantian Zhang","Xueqian Wang","Bin Liang","Bo Yuan"],"abstract":"The powerful learning ability of deep neural networks enables reinforcement learning agents to learn competent control policies directly from continuous environments. In theory, to achieve stable performance, neural networks assume i.i.d. inputs, which unfortunately does no hold in the general reinforcement learning paradigm where the training data is temporally correlated and non-stationary. This issue may lead to the phenomenon of \"catastrophic interference\" and the collapse in performance. In this paper, we present IQ, i.e., interference-aware deep Q-learning, to mitigate catastrophic interference in single-task deep reinforcement learning. Specifically, we resort to online clustering to achieve on-the-fly context division, together with a multi-head network and a knowledge distillation regularization term for preserving the policy of learned contexts. Built upon deep Q networks, IQ consistently boosts the stability and performance when compared to existing methods, verified with extensive experiments on classic control and Atari tasks. The code is publicly available at: https://github.com/Sweety-dm/Interference-aware-Deep-Q-learning.","url_abs":"https://arxiv.org/abs/2109.00525v2","url_pdf":"https://arxiv.org/pdf/2109.00525v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"catastrophic-interference-in-reinforcement","repo_url":"https://github.com/sweety-dm/interference-aware-deep-q-learning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"general-reinforcement-learning","task_name":"General Reinforcement Learning"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"online-clustering","task_name":"Online Clustering"},{"task_slug":"openai-gym","task_name":"OpenAI Gym"},{"task_slug":"q-learning","task_name":"Q-Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2109.00525","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2109.00525"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sweety-dm/interference-aware-deep-q-learning","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"38f7f3502614c033","entry":"context_decision","repo":"sweety-dm/interference-aware-deep-q-learning","repo_kind":"official","path":"IQ-RE/IQ_dqn_agent.py","file_url":"https://github.com/sweety-dm/interference-aware-deep-q-learning/blob/HEAD/IQ-RE/IQ_dqn_agent.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"38f7f3502614c033"}},{"code_sha256_prefix":"9b35c6236c98603f","entry":"current_state_head","repo":"sweety-dm/interference-aware-deep-q-learning","repo_kind":"official","path":"IQ-RE/IQ_dqn_agent.py","file_url":"https://github.com/sweety-dm/interference-aware-deep-q-learning/blob/HEAD/IQ-RE/IQ_dqn_agent.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b35c6236c98603f"}},{"code_sha256_prefix":"b92cca6f5c9b2346","entry":"head_others","repo":"sweety-dm/interference-aware-deep-q-learning","repo_kind":"official","path":"IQ-RE/IQ_dqn_agent.py","file_url":"https://github.com/sweety-dm/interference-aware-deep-q-learning/blob/HEAD/IQ-RE/IQ_dqn_agent.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b92cca6f5c9b2346"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}