{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gradient-trained-weights-in-wide-neural","title":"How to Train Your Wide Neural Network Without Backprop: An Input-Weight Alignment Perspective","arxiv_id":"2106.08453","date":"2021-06-15","proceeding":null,"authors":["Akhilan Boopathy","Ila Fiete"],"abstract":"Recent works have examined theoretical and empirical properties of wide neural networks trained in the Neural Tangent Kernel (NTK) regime. Given that biological neural networks are much wider than their artificial counterparts, we consider NTK regime wide neural networks as a possible model of biological neural networks. Leveraging NTK theory, we show theoretically that gradient descent drives layerwise weight updates that are aligned with their input activity correlations weighted by error, and demonstrate empirically that the result also holds in finite-width wide networks. The alignment result allows us to formulate a family of biologically-motivated, backpropagation-free learning rules that are theoretically equivalent to backpropagation in infinite-width networks. We test these learning rules on benchmark problems in feedforward and recurrent neural networks and demonstrate, in wide networks, comparable performance to backpropagation. The proposed rules are particularly effective in low data regimes, which are common in biological learning settings.","url_abs":"https://arxiv.org/abs/2106.08453v2","url_pdf":"https://arxiv.org/pdf/2106.08453v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gradient-trained-weights-in-wide-neural","repo_url":"https://github.com/fietelab/wide-network-alignment","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.08453","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.08453"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/fietelab/wide-network-alignment","reach":null}],"summary":{"ran_violates":1,"ran_draft_wrong":1,"ran_honours":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"eef8b3bc5670f807","entry":"flatten","repo":"fietelab/wide-network-alignment","repo_kind":"official","path":"alignment_cifar.py","file_url":"https://github.com/fietelab/wide-network-alignment/blob/HEAD/alignment_cifar.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eef8b3bc5670f807"}},{"code_sha256_prefix":"6934168c2cba3c37","entry":"load","repo":"fietelab/wide-network-alignment","repo_kind":"official","path":"alignment_cifar.py","file_url":"https://github.com/fietelab/wide-network-alignment/blob/HEAD/alignment_cifar.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6934168c2cba3c37"}},{"code_sha256_prefix":"169575fc2515e3b6","entry":"shuffle","repo":"fietelab/wide-network-alignment","repo_kind":"official","path":"alignment_cifar.py","file_url":"https://github.com/fietelab/wide-network-alignment/blob/HEAD/alignment_cifar.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"169575fc2515e3b6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}