{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-theoretical-framework-for-target","title":"A Theoretical Framework for Target Propagation","arxiv_id":"2006.14331","date":"2020-06-25","proceeding":"NeurIPS 2020 12","authors":["Alexander Meulemans","Francesco S. Carzaniga","Johan A. K. Suykens","João Sacramento","Benjamin F. Grewe"],"abstract":"The success of deep learning, a brain-inspired form of AI, has sparked interest in understanding how the brain could similarly learn across multiple layers of neurons. However, the majority of biologically-plausible learning algorithms have not yet reached the performance of backpropagation (BP), nor are they built on strong theoretical foundations. Here, we analyze target propagation (TP), a popular but not yet fully understood alternative to BP, from the standpoint of mathematical optimization. Our theory shows that TP is closely related to Gauss-Newton optimization and thus substantially differs from BP. Furthermore, our analysis reveals a fundamental limitation of difference target propagation (DTP), a well-known variant of TP, in the realistic scenario of non-invertible neural networks. We provide a first solution to this problem through a novel reconstruction loss that improves feedback weight training, while simultaneously introducing architectural flexibility by allowing for direct feedback connections from the output to each hidden layer. Our theory is corroborated by experimental results that show significant improvements in performance and in the alignment of forward weight updates with loss gradients, compared to DTP.","url_abs":"https://arxiv.org/abs/2006.14331v4","url_pdf":"https://arxiv.org/pdf/2006.14331v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-theoretical-framework-for-target","repo_url":"https://github.com/AlexanderMeulemans/TheoreticalFrameworkForTargetPropagation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"a-theoretical-framework-for-target","repo_url":"https://github.com/meulemansalex/theoretical_framework_for_target_propagation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2006.14331","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2006.14331"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/meulemansalex/theoretical_framework_for_target_propagation","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/AlexanderMeulemans/TheoreticalFrameworkForTargetPropagation","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d38d8a0b7b053904","entry":"accuracy","repo":"AlexanderMeulemans/TheoreticalFrameworkForTargetPropagation","repo_kind":"official","path":"lib/utils.py","file_url":"https://github.com/AlexanderMeulemans/TheoreticalFrameworkForTargetPropagation/blob/HEAD/lib/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d38d8a0b7b053904"}},{"code_sha256_prefix":"b256dc876fd34c06","entry":"choose_forward_optimizer","repo":"AlexanderMeulemans/TheoreticalFrameworkForTargetPropagation","repo_kind":"official","path":"lib/utils.py","file_url":"https://github.com/AlexanderMeulemans/TheoreticalFrameworkForTargetPropagation/blob/HEAD/lib/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b256dc876fd34c06"}},{"code_sha256_prefix":"da6c0d07f0a5cfc2","entry":"choose_optimizer","repo":"AlexanderMeulemans/TheoreticalFrameworkForTargetPropagation","repo_kind":"official","path":"lib/utils.py","file_url":"https://github.com/AlexanderMeulemans/TheoreticalFrameworkForTargetPropagation/blob/HEAD/lib/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"da6c0d07f0a5cfc2"}},{"code_sha256_prefix":"c3f6ba0f418c116c","entry":"generate_data_from_teacher_network","repo":"AlexanderMeulemans/TheoreticalFrameworkForTargetPropagation","repo_kind":"official","path":"lib/builders.py","file_url":"https://github.com/AlexanderMeulemans/TheoreticalFrameworkForTargetPropagation/blob/HEAD/lib/builders.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c3f6ba0f418c116c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}