{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/spike-based-residual-blocks","title":"Deep Residual Learning in Spiking Neural Networks","arxiv_id":"2102.04159","date":"2021-02-08","proceeding":"NeurIPS 2021 12","authors":["Wei Fang","Zhaofei Yu","Yanqi Chen","Tiejun Huang","Timothée Masquelier","Yonghong Tian"],"abstract":"Deep Spiking Neural Networks (SNNs) present optimization difficulties for gradient-based approaches due to discrete binary activation and complex spatial-temporal dynamics. Considering the huge success of ResNet in deep learning, it would be natural to train deep SNNs with residual learning. Previous Spiking ResNet mimics the standard residual block in ANNs and simply replaces ReLU activation layers with spiking neurons, which suffers the degradation problem and can hardly implement residual learning. In this paper, we propose the spike-element-wise (SEW) ResNet to realize residual learning in deep SNNs. We prove that the SEW ResNet can easily implement identity mapping and overcome the vanishing/exploding gradient problems of Spiking ResNet. We evaluate our SEW ResNet on ImageNet, DVS Gesture, and CIFAR10-DVS datasets, and show that SEW ResNet outperforms the state-of-the-art directly trained SNNs in both accuracy and time-steps. Moreover, SEW ResNet can achieve higher performance by simply adding more layers, providing a simple method to train deep SNNs. To our best knowledge, this is the first time that directly training deep SNNs with more than 100 layers becomes possible. Our codes are available at https://github.com/fangwei123456/Spike-Element-Wise-ResNet.","url_abs":"https://arxiv.org/abs/2102.04159v6","url_pdf":"https://arxiv.org/pdf/2102.04159v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"spike-based-residual-blocks","repo_url":"https://github.com/fangwei123456/Spike-Element-Wise-ResNet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2102.04159","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2102.04159"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/fangwei123456/Spike-Element-Wise-ResNet","reach":{"status":"unanswered"}}],"summary":{"ran_draft_wrong":1,"ran_fixture":1,"unverified":1},"by_repo_kind":{"community":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"35d4b6c240682f59","entry":"parse_source_specs","repo":"fangwei123456/spikingjelly","repo_kind":"community","path":"benchmark/benchmark_snn_single_gpu.py","file_url":"https://github.com/fangwei123456/spikingjelly/blob/HEAD/benchmark/benchmark_snn_single_gpu.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OPAQUE_PARAMS","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"35d4b6c240682f59"}},{"code_sha256_prefix":"0e7fc0adcde2187a","entry":"summarize_samples","repo":"fangwei123456/spikingjelly","repo_kind":"community","path":"benchmark/benchmark_snn_single_gpu.py","file_url":"https://github.com/fangwei123456/spikingjelly/blob/HEAD/benchmark/benchmark_snn_single_gpu.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"0e7fc0adcde2187a"}},{"code_sha256_prefix":"d9a4a9379d102155","entry":"case_key","repo":"fangwei123456/spikingjelly","repo_kind":"community","path":"benchmark/benchmark_snn_single_gpu.py","file_url":"https://github.com/fangwei123456/spikingjelly/blob/HEAD/benchmark/benchmark_snn_single_gpu.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"d9a4a9379d102155"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}