{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/temporal-efficient-training-of-spiking-neural-1","title":"Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting","arxiv_id":"2202.11946","date":"2022-02-24","proceeding":"ICLR 2022 4","authors":["Shikuang Deng","Yuhang Li","Shanghang Zhang","Shi Gu"],"abstract":"Recently, brain-inspired spiking neuron networks (SNNs) have attracted widespread research interest because of their event-driven and energy-efficient characteristics. Still, it is difficult to efficiently train deep SNNs due to the non-differentiability of its activation function, which disables the typically used gradient descent approaches for traditional artificial neural networks (ANNs). Although the adoption of surrogate gradient (SG) formally allows for the back-propagation of losses, the discrete spiking mechanism actually differentiates the loss landscape of SNNs from that of ANNs, failing the surrogate gradient methods to achieve comparable accuracy as for ANNs. In this paper, we first analyze why the current direct training approach with surrogate gradient results in SNNs with poor generalizability. Then we introduce the temporal efficient training (TET) approach to compensate for the loss of momentum in the gradient descent with SG so that the training process can converge into flatter minima with better generalizability. Meanwhile, we demonstrate that TET improves the temporal scalability of SNN and induces a temporal inheritable training for acceleration. Our method consistently outperforms the SOTA on all reported mainstream datasets, including CIFAR-10/100 and ImageNet. Remarkably on DVS-CIFAR10, we obtained 83$\\%$ top-1 accuracy, over 10$\\%$ improvement compared to existing state of the art. Codes are available at \\url{https://github.com/Gus-Lab/temporal_efficient_training}.","url_abs":"https://arxiv.org/abs/2202.11946v3","url_pdf":"https://arxiv.org/pdf/2202.11946v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"temporal-efficient-training-of-spiking-neural-1","repo_url":"https://github.com/gus-lab/temporal_efficient_training","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2202.11946","atlas_url":"https://app.syntology.ai/?focus=2202.11946","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2202.11946"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Gus-Lab/temporal_","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gus-lab/temporal_efficient_training","reach":null}],"summary":{"ran_draft_wrong":3,"ran_honours":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1},"community":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"027e234d17d12457","entry":"normalizex","repo":"brain-intelligence-lab/temporal_efficient_training","repo_kind":"community","path":"models/layers.py","file_url":"https://github.com/brain-intelligence-lab/temporal_efficient_training/blob/HEAD/models/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"027e234d17d12457"}},{"code_sha256_prefix":"e63c09a8d741cb95","entry":"TET_loss","repo":"gus-lab/temporal_efficient_training","repo_kind":"official","path":"functions.py","file_url":"https://github.com/gus-lab/temporal_efficient_training/blob/HEAD/functions.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e63c09a8d741cb95"}},{"code_sha256_prefix":"685a0742dceb815a","entry":"add_dimention","repo":"brain-intelligence-lab/temporal_efficient_training","repo_kind":"community","path":"models/layers.py","file_url":"https://github.com/brain-intelligence-lab/temporal_efficient_training/blob/HEAD/models/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"685a0742dceb815a"}},{"code_sha256_prefix":"0f238aceb8acee39","entry":"test","repo":"brain-intelligence-lab/temporal_efficient_training","repo_kind":"community","path":"main_training_parallel.py","file_url":"https://github.com/brain-intelligence-lab/temporal_efficient_training/blob/HEAD/main_training_parallel.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f238aceb8acee39"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}