{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-theoretical-framework-for-inference-and","title":"A Theoretical Framework for Inference and Learning in Predictive Coding Networks","arxiv_id":"2207.12316","date":"2022-07-21","proceeding":null,"authors":["Beren Millidge","Yuhang Song","Tommaso Salvatori","Thomas Lukasiewicz","Rafal Bogacz"],"abstract":"Predictive coding (PC) is an influential theory in computational neuroscience, which argues that the cortex forms unsupervised world models by implementing a hierarchical process of prediction error minimization. PC networks (PCNs) are trained in two phases. First, neural activities are updated to optimize the network's response to external stimuli. Second, synaptic weights are updated to consolidate this change in activity -- an algorithm called \\emph{prospective configuration}. While previous work has shown how in various limits, PCNs can be found to approximate backpropagation (BP), recent work has demonstrated that PCNs operating in this standard regime, which does not approximate BP, nevertheless obtain competitive training and generalization performance to BP-trained networks while outperforming them on tasks such as online, few-shot, and continual learning, where brains are known to excel. Despite this promising empirical performance, little is understood theoretically about the properties and dynamics of PCNs in this regime. In this paper, we provide a comprehensive theoretical analysis of the properties of PCNs trained with prospective configuration. We first derive analytical results concerning the inference equilibrium for PCNs and a previously unknown close connection relationship to target propagation (TP). Secondly, we provide a theoretical analysis of learning in PCNs as a variant of generalized expectation-maximization and use that to prove the convergence of PCNs to critical points of the BP loss function, thus showing that deep PCNs can, in theory, achieve the same generalization performance as BP, while maintaining their unique advantages.","url_abs":"https://arxiv.org/abs/2207.12316v2","url_pdf":"https://arxiv.org/pdf/2207.12316v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-theoretical-framework-for-inference-and","repo_url":"https://github.com/BerenMillidge/theoretical_framework_predictive_coding","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"}],"methods":[{"method_slug":"pc","method_name":"pc"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2207.12316","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2207.12316"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/BerenMillidge/theoretical_framework_predictive_coding","reach":null}],"summary":{"ran_honours":2,"ran_fixture":2,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"fba8fe9552e1e4d9","entry":"accuracy","repo":"BerenMillidge/theoretical_framework_predictive_coding","repo_kind":"official","path":"pcn_learning.py","file_url":"https://github.com/BerenMillidge/theoretical_framework_predictive_coding/blob/HEAD/pcn_learning.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fba8fe9552e1e4d9"}},{"code_sha256_prefix":"6dc29ca9372db5fe","entry":"cosine_similarity","repo":"BerenMillidge/theoretical_framework_predictive_coding","repo_kind":"official","path":"four_layer_convergence.py","file_url":"https://github.com/BerenMillidge/theoretical_framework_predictive_coding/blob/HEAD/four_layer_convergence.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6dc29ca9372db5fe"}},{"code_sha256_prefix":"c3099d14671b3a91","entry":"linear_base_function","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"c3099d14671b3a91"}},{"code_sha256_prefix":"75839320898e351d","entry":"multi_layer_input_unconstrained_linear","repo":"BerenMillidge/theoretical_framework_predictive_coding","repo_kind":"official","path":"numerical_experiments.py","file_url":"https://github.com/BerenMillidge/theoretical_framework_predictive_coding/blob/HEAD/numerical_experiments.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"75839320898e351d"}},{"code_sha256_prefix":"161bf2f78bef02d3","entry":"verify_linear_equilibrium","repo":"BerenMillidge/theoretical_framework_predictive_coding","repo_kind":"official","path":"numerical_experiments.py","file_url":"https://github.com/BerenMillidge/theoretical_framework_predictive_coding/blob/HEAD/numerical_experiments.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"161bf2f78bef02d3"}},{"code_sha256_prefix":"e3ad41f2da07bd55","entry":"input_unconstrained_nonlinear","repo":"BerenMillidge/theoretical_framework_predictive_coding","repo_kind":"official","path":"numerical_experiments.py","file_url":"https://github.com/BerenMillidge/theoretical_framework_predictive_coding/blob/HEAD/numerical_experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e3ad41f2da07bd55"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}