{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/long-short-term-memory-and-learning-to-learn","title":"Long short-term memory and learning-to-learn in networks of spiking neurons","arxiv_id":"1803.09574","date":"2018-03-26","proceeding":"NeurIPS 2018 12","authors":["Guillaume Bellec","Darjan Salaj","Anand Subramoney","Robert Legenstein","Wolfgang Maass"],"abstract":"Recurrent networks of spiking neurons (RSNNs) underlie the astounding\ncomputing and learning capabilities of the brain. But computing and learning\ncapabilities of RSNN models have remained poor, at least in comparison with\nartificial neural networks (ANNs). We address two possible reasons for that.\nOne is that RSNNs in the brain are not randomly connected or designed according\nto simple rules, and they do not start learning as a tabula rasa network.\nRather, RSNNs in the brain were optimized for their tasks through evolution,\ndevelopment, and prior experience. Details of these optimization processes are\nlargely unknown. But their functional contribution can be approximated through\npowerful optimization methods, such as backpropagation through time (BPTT).\n  A second major mismatch between RSNNs in the brain and models is that the\nlatter only show a small fraction of the dynamics of neurons and synapses in\nthe brain. We include neurons in our RSNN model that reproduce one prominent\ndynamical process of biological neurons that takes place at the behaviourally\nrelevant time scale of seconds: neuronal adaptation. We denote these networks\nas LSNNs because of their Long short-term memory. The inclusion of adapting\nneurons drastically increases the computing and learning capability of RSNNs if\nthey are trained and configured by deep learning (BPTT combined with a rewiring\nalgorithm that optimizes the network architecture). In fact, the computational\nperformance of these RSNNs approaches for the first time that of LSTM networks.\nIn addition RSNNs with adapting neurons can acquire abstract knowledge from\nprior learning in a Learning-to-Learn (L2L) scheme, and transfer that knowledge\nin order to learn new but related tasks from very few examples. We demonstrate\nthis for supervised learning and reinforcement learning.","url_abs":"http://arxiv.org/abs/1803.09574v4","url_pdf":"http://arxiv.org/pdf/1803.09574v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"long-short-term-memory-and-learning-to-learn","repo_url":"https://github.com/IGITUGraz/LSNN-official","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"long-short-term-memory-and-learning-to-learn","repo_url":"https://github.com/norse/norse","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"sequential-image-classification","task_name":"Sequential Image Classification"},{"task_slug":"speech-recognition","task_name":"Speech Recognition"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/speech-recognition-on-timit","task":"Speech Recognition","dataset":"TIMIT","model":"LSNN","rank_in_archive_order":22,"of":22,"metrics":{"Percentage error":"33.2"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1803.09574","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1803.09574"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/norse/norse","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/IGITUGraz/LSNN-official","reach":null}],"summary":{"ran_draft_wrong":2,"ran_fixture":1,"unverified":2},"by_repo_kind":{"listed":{"samples":5,"ran":3,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"19bc8633487d68c3","entry":"feed_dict_with_placeholder_container","repo":"IGITUGraz/LSNN-official","repo_kind":"listed","path":"lsnn/spiking_models.py","file_url":"https://github.com/IGITUGraz/LSNN-official/blob/HEAD/lsnn/spiking_models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"BSD-3-Clause-Clear","inline_ok":false,"mcp_get_code":{"code_sha256":"19bc8633487d68c3"}},{"code_sha256_prefix":"43be13e0cdaf00e2","entry":"find_onset_offset","repo":"IGITUGraz/LSNN-official","repo_kind":"listed","path":"bin/tutorial_sequential_mnist_with_LSNN.py","file_url":"https://github.com/IGITUGraz/LSNN-official/blob/HEAD/bin/tutorial_sequential_mnist_with_LSNN.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"BSD-3-Clause-Clear","inline_ok":false,"mcp_get_code":{"code_sha256":"43be13e0cdaf00e2"}},{"code_sha256_prefix":"279c5040e23318aa","entry":"load","repo":"norse/norse","repo_kind":"listed","path":"norse/task/mnist.py","file_url":"https://github.com/norse/norse/blob/HEAD/norse/task/mnist.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"LGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"279c5040e23318aa"}},{"code_sha256_prefix":"9965114171da1c70","entry":"test","repo":"norse/norse","repo_kind":"listed","path":"norse/task/mnist.py","file_url":"https://github.com/norse/norse/blob/HEAD/norse/task/mnist.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"9965114171da1c70"}},{"code_sha256_prefix":"daa1278e5382ed85","entry":"train","repo":"norse/norse","repo_kind":"listed","path":"norse/task/mnist.py","file_url":"https://github.com/norse/norse/blob/HEAD/norse/task/mnist.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"daa1278e5382ed85"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}