{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/recurrent-linear-transformers","title":"AGaLiTe: Approximate Gated Linear Transformers for Online Reinforcement Learning","arxiv_id":"2310.15719","date":"2023-10-24","proceeding":null,"authors":["Subhojeet Pramanik","Esraa Elelimy","Marlos C. Machado","Adam White"],"abstract":"In this paper we investigate transformer architectures designed for partially observable online reinforcement learning. The self-attention mechanism in the transformer architecture is capable of capturing long-range dependencies and it is the main reason behind its effectiveness in processing sequential data. Nevertheless, despite their success, transformers have two significant drawbacks that still limit their applicability in online reinforcement learning: (1) in order to remember all past information, the self-attention mechanism requires access to the whole history to be provided as context. (2) The inference cost in transformers is expensive. In this paper, we introduce recurrent alternatives to the transformer self-attention mechanism that offer context-independent inference cost, leverage long-range dependencies effectively, and performs well in online reinforcement learning task. We quantify the impact of the different components of our architecture in a diagnostic environment and assess performance gains in 2D and 3D pixel-based partially-observable environments (e.g. T-Maze, Mystery Path, Craftax, and Memory Maze). Compared with a state-of-the-art architecture, GTrXL, inference in our approach is at least 40% cheaper while reducing memory use more than 50%. Our approach either performs similarly or better than GTrXL, improving more than 37% upon GTrXL performance in harder tasks.","url_abs":"https://arxiv.org/abs/2310.15719v2","url_pdf":"https://arxiv.org/pdf/2310.15719v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"recurrent-linear-transformers","repo_url":"https://github.com/subho406/Recurrent-Linear-Transformers","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"recurrent-linear-transformers","repo_url":"https://github.com/subho406/agalite","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"jax","reach":null}],"tasks":[{"task_slug":"diagnostic","task_name":"Diagnostic"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"gru","method_name":"GRU"},{"method_slug":"gtrxl","method_name":"GTrXL"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.15719","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.15719"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/subho406/agalite","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/subho406/Recurrent-Linear-Transformers","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4,"ran_fixture":1,"ran_draft_wrong":1,"ran_honours":1,"unverified":4},"by_repo_kind":{"official":{"samples":11,"ran":7,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3794764640830e25","entry":"actor_model_discete","repo":"subho406/Recurrent-Linear-Transformers","repo_kind":"official","path":"src/model_fns/achead_fns.py","file_url":"https://github.com/subho406/Recurrent-Linear-Transformers/blob/HEAD/src/model_fns/achead_fns.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3794764640830e25"}},{"code_sha256_prefix":"ed42d985a78a9d09","entry":"binary_operator","repo":"subho406/agalite","repo_kind":"official","path":"src/models/agalite/agalite.py","file_url":"https://github.com/subho406/agalite/blob/HEAD/src/models/agalite/agalite.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ed42d985a78a9d09"}},{"code_sha256_prefix":"85f1e371fbe56d2c","entry":"critic_model","repo":"subho406/Recurrent-Linear-Transformers","repo_kind":"official","path":"src/model_fns/achead_fns.py","file_url":"https://github.com/subho406/Recurrent-Linear-Transformers/blob/HEAD/src/model_fns/achead_fns.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"85f1e371fbe56d2c"}},{"code_sha256_prefix":"93e29f94d7e078ff","entry":"dpfp","repo":"subho406/agalite","repo_kind":"official","path":"src_pure/models/agalite.py","file_url":"https://github.com/subho406/agalite/blob/HEAD/src_pure/models/agalite.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"93e29f94d7e078ff"}},{"code_sha256_prefix":"99c1d0c4eabda3cc","entry":"jax_pad","repo":"subho406/Recurrent-Linear-Transformers","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/subho406/Recurrent-Linear-Transformers/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"99c1d0c4eabda3cc"}},{"code_sha256_prefix":"4ebe8a74c4714481","entry":"kernel_config_to_output_dim","repo":"subho406/agalite","repo_kind":"official","path":"src_pure/models/agalite.py","file_url":"https://github.com/subho406/agalite/blob/HEAD/src_pure/models/agalite.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4ebe8a74c4714481"}},{"code_sha256_prefix":"04c4ac86fa07622e","entry":"masked_fill","repo":"subho406/Recurrent-Linear-Transformers","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/subho406/Recurrent-Linear-Transformers/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"04c4ac86fa07622e"}},{"code_sha256_prefix":"89290db1912cbf52","entry":"discounted_sum_jax","repo":"subho406/agalite","repo_kind":"official","path":"src/models/agalite/agalite.py","file_url":"https://github.com/subho406/agalite/blob/HEAD/src/models/agalite/agalite.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"89290db1912cbf52"}},{"code_sha256_prefix":"52da6b9a0a2cbffe","entry":"discounted_sum_parallel","repo":"subho406/agalite","repo_kind":"official","path":"src/models/agalite/agalite.py","file_url":"https://github.com/subho406/agalite/blob/HEAD/src/models/agalite/agalite.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"52da6b9a0a2cbffe"}},{"code_sha256_prefix":"10be38ccb17045ea","entry":"main","repo":"subho406/Recurrent-Linear-Transformers","repo_kind":"official","path":"trainer_pure.py","file_url":"https://github.com/subho406/Recurrent-Linear-Transformers/blob/HEAD/trainer_pure.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"10be38ccb17045ea"}},{"code_sha256_prefix":"c63da903ea6731df","entry":"tree_dot","repo":"subho406/Recurrent-Linear-Transformers","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/subho406/Recurrent-Linear-Transformers/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c63da903ea6731df"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}