{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2603-11395","title":"ARROW: Augmented Replay for RObust World models","arxiv_id":"2603.11395","date":"2026-03-12","proceeding":null,"authors":["Abdulaziz Alyahya","Abdallah Al Siyabi","Markus R. Ernst","Luke Yang","Levin Kuhlmann","Gideon Kowadlo"],"abstract":"Continual reinforcement learning challenges agents to acquire new skills while retaining previously learned ones with the goal of improving performance in both past and future tasks. Most existing approaches rely on model-free methods with replay buffers to mitigate catastrophic forgetting; however, these solutions often face significant scalability challenges due to large memory demands. Drawing inspiration from neuroscience, where the brain replays experiences to a predictive World Model rather than directly to the policy, we present ARROW (Augmented Replay for RObust World models), a model-based continual RL algorithm that extends DreamerV3 with a memory-efficient, distribution-matching replay buffer. Unlike standard fixed-size FIFO buffers, ARROW maintains two complementary buffers: a short-term buffer for recent experiences and a long-term buffer that preserves task diversity through intelligent sampling. We evaluate ARROW on two challenging continual RL settings: Tasks without shared structure (Atari), and tasks with shared structure, where knowledge transfer is possible (Procgen CoinRun variants). Compared to model-free and model-based baselines with replay buffers of the same-size, ARROW demonstrates substantially less forgetting on tasks without shared structure, while maintaining comparable forward transfer. Our findings highlight the potential of model-based RL and bio-inspired approaches for continual reinforcement learning, warranting further research.","url_abs":"https://arxiv.org/abs/2603.11395","url_pdf":"https://arxiv.org/pdf/2603.11395","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2603.11395","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2603.11395"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/danijar/dreamerv3","reach":null},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/Cerenaut/ARROW","reach":null}],"summary":{"ran":6,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":8,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"07c895c9b4541189","entry":"ChLayerNorm","repo":"Cerenaut/ARROW","repo_kind":"found_in_text","path":"Code/ARROW_and_DV3/Atari/rssm.py","file_url":"https://github.com/Cerenaut/ARROW/blob/HEAD/Code/ARROW_and_DV3/Atari/rssm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"07c895c9b4541189"}},{"code_sha256_prefix":"fee6a1ca9a08f6a6","entry":"Encoder","repo":"Cerenaut/ARROW","repo_kind":"found_in_text","path":"Code/ARROW_and_DV3/Atari/rssm.py","file_url":"https://github.com/Cerenaut/ARROW/blob/HEAD/Code/ARROW_and_DV3/Atari/rssm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fee6a1ca9a08f6a6"}},{"code_sha256_prefix":"7b705a8903768b38","entry":"LayerNormSiLU","repo":"Cerenaut/ARROW","repo_kind":"found_in_text","path":"Code/ARROW_and_DV3/Atari/rssm.py","file_url":"https://github.com/Cerenaut/ARROW/blob/HEAD/Code/ARROW_and_DV3/Atari/rssm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7b705a8903768b38"}},{"code_sha256_prefix":"b27df8a9be8556a6","entry":"Recurrent","repo":"Cerenaut/ARROW","repo_kind":"found_in_text","path":"Code/ARROW_and_DV3/Atari/rssm.py","file_url":"https://github.com/Cerenaut/ARROW/blob/HEAD/Code/ARROW_and_DV3/Atari/rssm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b27df8a9be8556a6"}},{"code_sha256_prefix":"0adc093f2e2cdf03","entry":"Representation","repo":"Cerenaut/ARROW","repo_kind":"found_in_text","path":"Code/ARROW_and_DV3/Atari/rssm.py","file_url":"https://github.com/Cerenaut/ARROW/blob/HEAD/Code/ARROW_and_DV3/Atari/rssm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0adc093f2e2cdf03"}},{"code_sha256_prefix":"385e472bca3a8b2c","entry":"Transition","repo":"Cerenaut/ARROW","repo_kind":"found_in_text","path":"Code/ARROW_and_DV3/Atari/rssm.py","file_url":"https://github.com/Cerenaut/ARROW/blob/HEAD/Code/ARROW_and_DV3/Atari/rssm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"385e472bca3a8b2c"}},{"code_sha256_prefix":"43ec7a7fea6a3b26","entry":"Rssm","repo":"Cerenaut/ARROW","repo_kind":"found_in_text","path":"Code/ARROW_and_DV3/Atari/rssm.py","file_url":"https://github.com/Cerenaut/ARROW/blob/HEAD/Code/ARROW_and_DV3/Atari/rssm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"43ec7a7fea6a3b26"}},{"code_sha256_prefix":"8e8726b5b2e4076e","entry":"get_mlp_layers","repo":"Cerenaut/ARROW","repo_kind":"found_in_text","path":"Code/ARROW_and_DV3/Atari/rssm.py","file_url":"https://github.com/Cerenaut/ARROW/blob/HEAD/Code/ARROW_and_DV3/Atari/rssm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8e8726b5b2e4076e"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}