{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/direct-feedback-alignment-provides-learning","title":"Direct Feedback Alignment Provides Learning in Deep Neural Networks","arxiv_id":"1609.01596","date":"2016-09-06","proceeding":"NeurIPS 2016 12","authors":["Arild Nøkland"],"abstract":"Artificial neural networks are most commonly trained with the\nback-propagation algorithm, where the gradient for learning is provided by\nback-propagating the error, layer by layer, from the output layer to the hidden\nlayers. A recently discovered method called feedback-alignment shows that the\nweights used for propagating the error backward don't have to be symmetric with\nthe weights used for propagation the activation forward. In fact, random\nfeedback weights work evenly well, because the network learns how to make the\nfeedback useful. In this work, the feedback alignment principle is used for\ntraining hidden layers more independently from the rest of the network, and\nfrom a zero initial condition. The error is propagated through fixed random\nfeedback connections directly from the output layer to each hidden layer. This\nsimple method is able to achieve zero training error even in convolutional\nnetworks and very deep networks, completely without error back-propagation. The\nmethod is a step towards biologically plausible machine learning because the\nerror signal is almost local, and no symmetric or reciprocal weights are\nrequired. Experiments show that the test performance on MNIST and CIFAR is\nalmost as good as those obtained with back-propagation for fully connected\nnetworks. If combined with dropout, the method achieves 1.45% error on the\npermutation invariant MNIST task.","url_abs":"http://arxiv.org/abs/1609.01596v5","url_pdf":"http://arxiv.org/pdf/1609.01596v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"direct-feedback-alignment-provides-learning","repo_url":"https://github.com/anokland/dfa-torch","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"torch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"direct-feedback-alignment-provides-learning","repo_url":"https://github.com/dbehrlich/directFeedbackAlignment","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"direct-feedback-alignment-provides-learning","repo_url":"https://github.com/florian-bacho/dirdfa","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"direct-feedback-alignment-provides-learning","repo_url":"https://github.com/jsalbert/biotorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"direct-feedback-alignment-provides-learning","repo_url":"https://github.com/metataro/DirectFeedbackAlignment","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"dfa-1","method_name":"DFA"}],"datasets_introduced":[],"methods_introduced":[{"slug":"dfa-1","name":"DFA","full_name":"Direct Feedback Alignment"}],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1609.01596","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1609.01596"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/anokland/dfa-torch","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/metataro/DirectFeedbackAlignment","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jsalbert/biotorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/florian-bacho/dirdfa","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dbehrlich/directFeedbackAlignment","reach":{"status":"ok"}}],"summary":{"unverified":3},"by_repo_kind":{"listed":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"58b2687e1bd48b65","entry":"functional_loss","repo":"florian-bacho/dirdfa","repo_kind":"listed","path":"main_fwd_grad.py","file_url":"https://github.com/florian-bacho/dirdfa/blob/HEAD/main_fwd_grad.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"58b2687e1bd48b65"}},{"code_sha256_prefix":"51c09bebc952acfe","entry":"test","repo":"florian-bacho/dirdfa","repo_kind":"listed","path":"main_fwd_grad.py","file_url":"https://github.com/florian-bacho/dirdfa/blob/HEAD/main_fwd_grad.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"51c09bebc952acfe"}},{"code_sha256_prefix":"8860f94661609337","entry":"train","repo":"florian-bacho/dirdfa","repo_kind":"listed","path":"main_fwd_grad.py","file_url":"https://github.com/florian-bacho/dirdfa/blob/HEAD/main_fwd_grad.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8860f94661609337"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}