{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2510-19732","title":"Memo: Training Memory-Efficient Embodied Agents with Reinforcement Learning","arxiv_id":"2510.19732","date":"2025-10-22","proceeding":"NeurIPS","authors":["Gunshi Gupta","Karmesh Yadav","Zsolt Kira","Yarin Gal","Rahaf Aljundi"],"abstract":"To enable embodied agents to operate effectively over extended timeframes, it is crucial to develop models that form and access memories to stay contextualized in their environment. In the current paradigm of training transformer-based policies for embodied sequential decision-making tasks, visual inputs often overwhelm the context limits of transformers, while humans can maintain and utilize a lifetime of experience compressed as memories. Significant compression is possible in principle, as much of the input is irrelevant and can be abstracted. However, existing approaches predominantly focus on either recurrent models with fixed-size memory or transformers with full-context reliance. In this work, we propose Memo, a transformer-based architecture and training recipe for reinforcement learning (RL) on memory-intensive, long-horizon tasks. Memo incorporates the creation and retrieval of memory by interleaving periodic summarization tokens with the inputs of a model during training. We demonstrate Memo's effectiveness on a gridworld meta-RL benchmark and a multi-object navigation task in photo-realistic indoor settings. Memo outperforms naive long-context transformer baselines while being more compute and storage efficient. Additionally, Memo generalizes better to longer contexts at inference time and remains robust in streaming settings, where historical context must be truncated to fit inference constraints. Our code is available at: https://github.com/gunshi/memo.","url_abs":"https://arxiv.org/abs/2510.19732","url_pdf":"https://arxiv.org/pdf/2510.19732","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2510.19732","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2510.19732"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/gunshi/memo","reach":{"status":"ok"}},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/UT-Austin-RPL/amago","reach":{"status":"ok","spdx":"MIT"}},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/Memory-icrl/memo","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":8},"by_repo_kind":{"found_in_text":{"samples":8,"ran":0,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"bd53f926adbfbdb4","entry":"ContinuousActionDist","repo":"Memory-icrl/memo","repo_kind":"found_in_text","path":"amago/nets/actor_critic.py","file_url":"https://github.com/Memory-icrl/memo/blob/HEAD/amago/nets/actor_critic.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bd53f926adbfbdb4"}},{"code_sha256_prefix":"a928ef5746d94b5c","entry":"DiscreteActionDist","repo":"Memory-icrl/memo","repo_kind":"found_in_text","path":"amago/nets/actor_critic.py","file_url":"https://github.com/Memory-icrl/memo/blob/HEAD/amago/nets/actor_critic.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a928ef5746d94b5c"}},{"code_sha256_prefix":"040df9379437c972","entry":"add_common_cli","repo":"Memory-icrl/memo","repo_kind":"found_in_text","path":"amago/cli_utils.py","file_url":"https://github.com/Memory-icrl/memo/blob/HEAD/amago/cli_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"040df9379437c972"}},{"code_sha256_prefix":"72b0399bfcee1e5c","entry":"create_episodes_mask","repo":"gunshi/memo","repo_kind":"found_in_text","path":"memo/policies/llamarl/modeling_llamarl.py","file_url":"https://github.com/gunshi/memo/blob/HEAD/memo/policies/llamarl/modeling_llamarl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"72b0399bfcee1e5c"}},{"code_sha256_prefix":"670359641d76db94","entry":"get_maze_obs_space","repo":"Memory-icrl/memo","repo_kind":"found_in_text","path":"amago/learning.py","file_url":"https://github.com/Memory-icrl/memo/blob/HEAD/amago/learning.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"670359641d76db94"}},{"code_sha256_prefix":"3d54758ed67ca815","entry":"get_wrapped_prop","repo":"gunshi/memo","repo_kind":"found_in_text","path":"memo/interactive_play.py","file_url":"https://github.com/gunshi/memo/blob/HEAD/memo/interactive_play.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3d54758ed67ca815"}},{"code_sha256_prefix":"02f1e778c5d0c0f7","entry":"reverse_hidden_window","repo":"gunshi/memo","repo_kind":"found_in_text","path":"memo/policies/cores.py","file_url":"https://github.com/gunshi/memo/blob/HEAD/memo/policies/cores.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"02f1e778c5d0c0f7"}},{"code_sha256_prefix":"3bb20f48b8e49d44","entry":"step_env","repo":"gunshi/memo","repo_kind":"found_in_text","path":"memo/interactive_play.py","file_url":"https://github.com/gunshi/memo/blob/HEAD/memo/interactive_play.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3bb20f48b8e49d44"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.AI","source":"arxiv_api"},"syntology_extracted_results":null}