{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-learning-with-asymmetric-connections-and","title":"Deep learning with asymmetric connections and Hebbian updates","arxiv_id":"1812.07965","date":"2018-11-19","proceeding":null,"authors":["Yali Amit"],"abstract":"We show that deep networks can be trained using Hebbian updates yielding\nsimilar performance to ordinary back-propagation on challenging image datasets.\nTo overcome the unrealistic symmetry in connections between layers, implicit in\nback-propagation, the feedback weights are separate from the feedforward\nweights. The feedback weights are also updated with a local rule, the same as\nthe feedforward weights - a weight is updated solely based on the product of\nactivity of the units it connects. With fixed feedback weights as proposed in\nLillicrap et. al (2016) performance degrades quickly as the depth of the\nnetwork increases. If the feedforward and feedback weights are initialized with\nthe same values, as proposed in Zipser and Rumelhart (1990), they remain the\nsame throughout training thus precisely implementing back-propagation. We show\nthat even when the weights are initialized differently and at random, and the\nalgorithm is no longer performing back-propagation, performance is comparable\non challenging datasets. We also propose a cost function whose derivative can\nbe represented as a local Hebbian update on the last layer. Convolutional\nlayers are updated with tied weights across space, which is not biologically\nplausible. We show that similar performance is achieved with untied layers,\nalso known as locally connected layers, corresponding to the connectivity\nimplied by the convolutional layers, but where weights are untied and updated\nseparately. In the linear case we show theoretically that the convergence of\nthe error to zero is accelerated by the update of the feedback weights.","url_abs":"http://arxiv.org/abs/1812.07965v2","url_pdf":"http://arxiv.org/pdf/1812.07965v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-learning-with-asymmetric-connections-and","repo_url":"https://github.com/yaliamit/URFB","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1812.07965","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1812.07965"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yaliamit/URFB","reach":null}],"summary":{"ran_violates":1,"ran_draft_wrong":2},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"d1a55ea3f29696e5","entry":"comp_lim","repo":"yaliamit/URFB","repo_kind":"official","path":"Conv_layers.py","file_url":"https://github.com/yaliamit/URFB/blob/HEAD/Conv_layers.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d1a55ea3f29696e5"}},{"code_sha256_prefix":"e214bb1158b3b54b","entry":"find_ts","repo":"yaliamit/URFB","repo_kind":"official","path":"Conv_net_gpu.py","file_url":"https://github.com/yaliamit/URFB/blob/HEAD/Conv_net_gpu.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e214bb1158b3b54b"}},{"code_sha256_prefix":"8af19aee7a6d0ee1","entry":"find_wr","repo":"yaliamit/URFB","repo_kind":"official","path":"Conv_net_gpu.py","file_url":"https://github.com/yaliamit/URFB/blob/HEAD/Conv_net_gpu.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8af19aee7a6d0ee1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}