{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/contrastive-hebbian-learning-with-random","title":"Contrastive Hebbian Learning with Random Feedback Weights","arxiv_id":"1806.07406","date":"2018-06-19","proceeding":null,"authors":["Georgios Detorakis","Travis Bartley","Emre Neftci"],"abstract":"Neural networks are commonly trained to make predictions through learning\nalgorithms. Contrastive Hebbian learning, which is a powerful rule inspired by\ngradient backpropagation, is based on Hebb's rule and the contrastive\ndivergence algorithm. It operates in two phases, the forward (or free) phase,\nwhere the data are fed to the network, and a backward (or clamped) phase, where\nthe target signals are clamped to the output layer of the network and the\nfeedback signals are transformed through the transpose synaptic weight\nmatrices. This implies symmetries at the synaptic level, for which there is no\nevidence in the brain. In this work, we propose a new variant of the algorithm,\ncalled random contrastive Hebbian learning, which does not rely on any synaptic\nweights symmetries. Instead, it uses random matrices to transform the feedback\nsignals during the clamped phase, and the neural dynamics are described by\nfirst order non-linear differential equations. The algorithm is experimentally\nverified by solving a Boolean logic task, classification tasks (handwritten\ndigits and letters), and an autoencoding task. This article also shows how the\nparameters affect learning, especially the random matrices. We use the\npseudospectra analysis to investigate further how random matrices impact the\nlearning process. Finally, we discuss the biological plausibility of the\nproposed algorithm, and how it can give rise to better computational models for\nlearning.","url_abs":"http://arxiv.org/abs/1806.07406v1","url_pdf":"http://arxiv.org/pdf/1806.07406v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"contrastive-hebbian-learning-with-random","repo_url":"https://github.com/gdetor/pygpsa","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}