{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scaling-spike-driven-transformer-with","title":"Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Training","arxiv_id":"2411.16061","date":"2024-11-25","proceeding":null,"authors":["Man Yao","Xuerui Qiu","Tianxiang Hu","Jiakui Hu","Yuhong Chou","Keyu Tian","Jianxing Liao","Luziwei Leng","Bo Xu","Guoqi Li"],"abstract":"The ambition of brain-inspired Spiking Neural Networks (SNNs) is to become a low-power alternative to traditional Artificial Neural Networks (ANNs). This work addresses two major challenges in realizing this vision: the performance gap between SNNs and ANNs, and the high training costs of SNNs. We identify intrinsic flaws in spiking neurons caused by binary firing mechanisms and propose a Spike Firing Approximation (SFA) method using integer training and spike-driven inference. This optimizes the spike firing pattern of spiking neurons, enhancing efficient training, reducing power consumption, improving performance, enabling easier scaling, and better utilizing neuromorphic chips. We also develop an efficient spike-driven Transformer architecture and a spike-masked autoencoder to prevent performance degradation during SNN scaling. On ImageNet-1k, we achieve state-of-the-art top-1 accuracy of 78.5\\%, 79.8\\%, 84.0\\%, and 86.2\\% with models containing 10M, 19M, 83M, and 173M parameters, respectively. For instance, the 10M model outperforms the best existing SNN by 7.2\\% on ImageNet, with training time acceleration and inference energy efficiency improved by 4.5$\\times$ and 3.9$\\times$, respectively. We validate the effectiveness and efficiency of the proposed method across various tasks, including object detection, semantic segmentation, and neuromorphic vision tasks. This work enables SNNs to match ANN performance while maintaining the low-power advantage, marking a significant step towards SNNs as a general visual backbone. Code is available at https://github.com/BICLab/Spike-Driven-Transformer-V3.","url_abs":"https://arxiv.org/abs/2411.16061v1","url_pdf":"https://arxiv.org/pdf/2411.16061v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scaling-spike-driven-transformer-with","repo_url":"https://github.com/biclab/spike-driven-transformer-v3","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"snn","method_name":"SNN"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2411.16061","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.16061"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/biclab/spike-driven-transformer-v3","reach":{"status":"ok"}}],"summary":{"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"57f12d460f0917e9","entry":"cal_acc","repo":"biclab/spike-driven-transformer-v3","repo_kind":"official","path":"SDT_V3/Classification/Model_Base/engine_finetune.py","file_url":"https://github.com/biclab/spike-driven-transformer-v3/blob/HEAD/SDT_V3/Classification/Model_Base/engine_finetune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"57f12d460f0917e9"}},{"code_sha256_prefix":"fed9681ff9f2423d","entry":"spike_conv1d","repo":"biclab/spike-driven-transformer-v3","repo_kind":"official","path":"SDT_V3/DVS/Hardvs/Qtrick_architecture/clock_driven/spike_op.py","file_url":"https://github.com/biclab/spike-driven-transformer-v3/blob/HEAD/SDT_V3/DVS/Hardvs/Qtrick_architecture/clock_driven/spike_op.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fed9681ff9f2423d"}},{"code_sha256_prefix":"7d240393806627b8","entry":"spike_conv2d","repo":"biclab/spike-driven-transformer-v3","repo_kind":"official","path":"SDT_V3/DVS/Hardvs/Qtrick_architecture/clock_driven/spike_op.py","file_url":"https://github.com/biclab/spike-driven-transformer-v3/blob/HEAD/SDT_V3/DVS/Hardvs/Qtrick_architecture/clock_driven/spike_op.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7d240393806627b8"}},{"code_sha256_prefix":"7712bfe8de3bbdd7","entry":"spike_linear","repo":"biclab/spike-driven-transformer-v3","repo_kind":"official","path":"SDT_V3/DVS/Hardvs/Qtrick_architecture/clock_driven/spike_op.py","file_url":"https://github.com/biclab/spike-driven-transformer-v3/blob/HEAD/SDT_V3/DVS/Hardvs/Qtrick_architecture/clock_driven/spike_op.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7712bfe8de3bbdd7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}