{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/temporally-efficient-deep-learning-with","title":"Temporally Efficient Deep Learning with Spikes","arxiv_id":"1706.04159","date":"2017-06-13","proceeding":"ICLR 2018 1","authors":["Peter O'Connor","Efstratios Gavves","Max Welling"],"abstract":"The vast majority of natural sensory data is temporally redundant. Video\nframes or audio samples which are sampled at nearby points in time tend to have\nsimilar values. Typically, deep learning algorithms take no advantage of this\nredundancy to reduce computation. This can be an obscene waste of energy. We\npresent a variant on backpropagation for neural networks in which computation\nscales with the rate of change of the data - not the rate at which we process\nthe data. We do this by having neurons communicate a combination of their\nstate, and their temporal change in state. Intriguingly, this simple\ncommunication rule give rise to units that resemble biologically-inspired leaky\nintegrate-and-fire neurons, and to a weight-update rule that is equivalent to a\nform of Spike-Timing Dependent Plasticity (STDP), a synaptic learning rule\nobserved in the brain. We demonstrate that on MNIST and a temporal variant of\nMNIST, our algorithm performs about as well as a Multilayer Perceptron trained\nwith backpropagation, despite only communicating discrete values between\nlayers.","url_abs":"http://arxiv.org/abs/1706.04159v1","url_pdf":"http://arxiv.org/pdf/1706.04159v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"temporally-efficient-deep-learning-with","repo_url":"https://github.com/petered/pdnn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1706.04159","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}