{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/equilibrium-propagation-bridging-the-gap","title":"Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation","arxiv_id":"1602.05179","date":"2016-02-16","proceeding":null,"authors":["Benjamin Scellier","Yoshua Bengio"],"abstract":"We introduce Equilibrium Propagation, a learning framework for energy-based\nmodels. It involves only one kind of neural computation, performed in both the\nfirst phase (when the prediction is made) and the second phase of training\n(after the target or prediction error is revealed). Although this algorithm\ncomputes the gradient of an objective function just like Backpropagation, it\ndoes not need a special computation or circuit for the second phase, where\nerrors are implicitly propagated. Equilibrium Propagation shares similarities\nwith Contrastive Hebbian Learning and Contrastive Divergence while solving the\ntheoretical issues of both algorithms: our algorithm computes the gradient of a\nwell defined objective function. Because the objective function is defined in\nterms of local perturbations, the second phase of Equilibrium Propagation\ncorresponds to only nudging the prediction (fixed point, or stationary\ndistribution) towards a configuration that reduces prediction error. In the\ncase of a recurrent multi-layer supervised network, the output units are\nslightly nudged towards their target in the second phase, and the perturbation\nintroduced at the output layer propagates backward in the hidden layers. We\nshow that the signal 'back-propagated' during this second phase corresponds to\nthe propagation of error derivatives and encodes the gradient of the objective\nfunction, when the synaptic update corresponds to a standard form of\nspike-timing dependent plasticity. This work makes it more plausible that a\nmechanism similar to Backpropagation could be implemented by brains, since\nleaky integrator neural computation performs both inference and error\nback-propagation in our model. The only local difference between the two phases\nis whether synaptic changes are allowed or not.","url_abs":"http://arxiv.org/abs/1602.05179v5","url_pdf":"http://arxiv.org/pdf/1602.05179v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"equilibrium-propagation-bridging-the-gap","repo_url":"https://github.com/bscellier/Towards-a-Biologically-Plausible-Backprop","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"equilibrium-propagation-bridging-the-gap","repo_url":"https://github.com/MatildeTristany/DEEP-model-ESANN-2020","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"prediction","task_name":"Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1602.05179","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}