{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/backpropagation-at-the-infinitesimal","title":"Backpropagation at the Infinitesimal Inference Limit of Energy-Based Models: Unifying Predictive Coding, Equilibrium Propagation, and Contrastive Hebbian Learning","arxiv_id":"2206.02629","date":"2022-05-31","proceeding":null,"authors":["Beren Millidge","Yuhang Song","Tommaso Salvatori","Thomas Lukasiewicz","Rafal Bogacz"],"abstract":"How the brain performs credit assignment is a fundamental unsolved problem in neuroscience. Many `biologically plausible' algorithms have been proposed, which compute gradients that approximate those computed by backpropagation (BP), and which operate in ways that more closely satisfy the constraints imposed by neural circuitry. Many such algorithms utilize the framework of energy-based models (EBMs), in which all free variables in the model are optimized to minimize a global energy function. However, in the literature, these algorithms exist in isolation and no unified theory exists linking them together. Here, we provide a comprehensive theory of the conditions under which EBMs can approximate BP, which lets us unify many of the BP approximation results in the literature (namely, predictive coding, equilibrium propagation, and contrastive Hebbian learning) and demonstrate that their approximation to BP arises from a simple and general mathematical property of EBMs at free-phase equilibrium. This property can then be exploited in different ways with different energy functions, and these specific choices yield a family of BP-approximating algorithms, which both includes the known results in the literature and can be used to derive new ones.","url_abs":"https://arxiv.org/abs/2206.02629v3","url_pdf":"https://arxiv.org/pdf/2206.02629v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"backpropagation-at-the-infinitesimal","repo_url":"https://github.com/BerenMillidge/infinitesimal_inference_limit","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2206.02629","atlas_url":"https://app.syntology.ai/?focus=2206.02629","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.02629"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/BerenMillidge/infinitesimal_inference_limit","reach":null}],"summary":{"ran_honours":2,"ran_draft_wrong":1,"ran_fixture":1},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"61343f9189262f12","entry":"accuracy","repo":"BerenMillidge/infinitesimal_inference_limit","repo_kind":"official","path":"pcn.py","file_url":"https://github.com/BerenMillidge/infinitesimal_inference_limit/blob/HEAD/pcn.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"61343f9189262f12"}},{"code_sha256_prefix":"7056daf18ffd93bb","entry":"get_pc_bp_params","repo":"BerenMillidge/infinitesimal_inference_limit","repo_kind":"official","path":"utils.py","file_url":"https://github.com/BerenMillidge/infinitesimal_inference_limit/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7056daf18ffd93bb"}},{"code_sha256_prefix":"c3099d14671b3a91","entry":"linear_base_function","repo":"BerenMillidge/infinitesimal_inference_limit","repo_kind":"official","path":"pcn.py","file_url":"https://github.com/BerenMillidge/infinitesimal_inference_limit/blob/HEAD/pcn.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c3099d14671b3a91"}},{"code_sha256_prefix":"ebe0a5b3f1d6acea","entry":"mse_loss","repo":"BerenMillidge/infinitesimal_inference_limit","repo_kind":"official","path":"utils.py","file_url":"https://github.com/BerenMillidge/infinitesimal_inference_limit/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ebe0a5b3f1d6acea"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}