{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/backprop-free-reinforcement-learning-with","title":"Backprop-Free Reinforcement Learning with Active Neural Generative Coding","arxiv_id":"2107.07046","date":"2021-07-10","proceeding":null,"authors":["Alexander Ororbia","Ankur Mali"],"abstract":"In humans, perceptual awareness facilitates the fast recognition and extraction of information from sensory input. This awareness largely depends on how the human agent interacts with the environment. In this work, we propose active neural generative coding, a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments. Specifically, we develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference. We demonstrate on several control problems, in the online learning setting, that our proposed modeling framework performs competitively with deep Q-learning models. The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.","url_abs":"https://arxiv.org/abs/2107.07046v1","url_pdf":"https://arxiv.org/pdf/2107.07046v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"backprop-free-reinforcement-learning-with","repo_url":"https://github.com/ago109/active-neural-generative-coding","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"jax","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"tasks":[{"task_slug":"q-learning","task_name":"Q-Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"q-learning","method_name":"Q-Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2107.07046","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2107.07046"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ago109/active-neural-generative-coding","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"summary":{"unverified":6},"by_repo_kind":{"official":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e497065188cde0a1","entry":"adjust_state","repo":"ago109/active-neural-generative-coding","repo_kind":"official","path":"src/model/ngc.py","file_url":"https://github.com/ago109/active-neural-generative-coding/blob/HEAD/src/model/ngc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"e497065188cde0a1"}},{"code_sha256_prefix":"1cd0959a32ed7d09","entry":"calc_mar_seq","repo":"ago109/active-neural-generative-coding","repo_kind":"official","path":"src/plot_reward.py","file_url":"https://github.com/ago109/active-neural-generative-coding/blob/HEAD/src/plot_reward.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"1cd0959a32ed7d09"}},{"code_sha256_prefix":"6041a9be022fdb64","entry":"calc_mar_window","repo":"ago109/active-neural-generative-coding","repo_kind":"official","path":"src/plot_reward.py","file_url":"https://github.com/ago109/active-neural-generative-coding/blob/HEAD/src/plot_reward.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"6041a9be022fdb64"}},{"code_sha256_prefix":"cc64ebc6add79763","entry":"calc_subwindow_avg","repo":"ago109/active-neural-generative-coding","repo_kind":"official","path":"src/plot_reward.py","file_url":"https://github.com/ago109/active-neural-generative-coding/blob/HEAD/src/plot_reward.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"cc64ebc6add79763"}},{"code_sha256_prefix":"d904b8e11c3aa97e","entry":"calc_syn_adjust","repo":"ago109/active-neural-generative-coding","repo_kind":"official","path":"src/model/ngc.py","file_url":"https://github.com/ago109/active-neural-generative-coding/blob/HEAD/src/model/ngc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"d904b8e11c3aa97e"}},{"code_sha256_prefix":"df62de02cb80cf8a","entry":"constrain","repo":"ago109/active-neural-generative-coding","repo_kind":"official","path":"src/model/ngc.py","file_url":"https://github.com/ago109/active-neural-generative-coding/blob/HEAD/src/model/ngc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"df62de02cb80cf8a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}