{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/spatial-aware-decision-making-with-ring","title":"Spatial-aware decision-making with ring attractors in reinforcement learning systems","arxiv_id":"2410.03119","date":"2024-10-04","proceeding":null,"authors":["Marcos Negre Saura","Richard Allmendinger","Theodore Papamarkou","Wei Pan"],"abstract":"This paper explores the integration of ring attractors, a mathematical model inspired by neural circuit dynamics, into the reinforcement learning (RL) action selection process. Ring attractors, as specialized brain-inspired structures that encode spatial information and uncertainty, offer a biologically plausible mechanism to improve learning speed and predictive performance. They do so by explicitly encoding the action space, facilitating the organization of neural activity, and enabling the distribution of spatial representations across the neural network in the context of deep RL. The application of ring attractors in the RL action selection process involves mapping actions to specific locations on the ring and decoding the selected action based on neural activity. We investigate the application of ring attractors by both building them as exogenous models and integrating them as part of a Deep Learning policy algorithm. Our results show a significant improvement in state-of-the-art models for the Atari 100k benchmark. Notably, our integrated approach improves the performance of state-of-the-art models by half, representing a 53\\% increase over selected baselines.","url_abs":"https://arxiv.org/abs/2410.03119v1","url_pdf":"https://arxiv.org/pdf/2410.03119v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.03119","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.03119"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/marcosaura/RA_RL","reach":{"status":"ok"}}],"summary":{"ran":2,"ran_fixture":1},"by_repo_kind":{"found_in_text":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"347b76ed10742d2a","entry":"concat_output","repo":"marcosaura/RA_RL","repo_kind":"found_in_text","path":"DL-RNN/EfficientZeroRA/core/model.py","file_url":"https://github.com/marcosaura/RA_RL/blob/HEAD/DL-RNN/EfficientZeroRA/core/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"347b76ed10742d2a"}},{"code_sha256_prefix":"05286bc5e4e73ead","entry":"concat_output_value","repo":"marcosaura/RA_RL","repo_kind":"found_in_text","path":"DL-RNN/EfficientZeroRA/core/model.py","file_url":"https://github.com/marcosaura/RA_RL/blob/HEAD/DL-RNN/EfficientZeroRA/core/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"05286bc5e4e73ead"}},{"code_sha256_prefix":"fa63400bb2942968","entry":"renormalize","repo":"marcosaura/RA_RL","repo_kind":"found_in_text","path":"DL-RNN/EfficientZeroRA/core/model.py","file_url":"https://github.com/marcosaura/RA_RL/blob/HEAD/DL-RNN/EfficientZeroRA/core/model.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fa63400bb2942968"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}