{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/temporal-separation-with-entropy","title":"Temporal Separation with Entropy Regularization for Knowledge Distillation in Spiking Neural Networks","arxiv_id":"2503.03144","date":"2025-03-05","proceeding":"CVPR 2025 1","authors":["Kairong Yu","Chengting Yu","Tianqing Zhang","Xiaochen Zhao","Shu Yang","Hongwei Wang","Qiang Zhang","Qi Xu"],"abstract":"Spiking Neural Networks (SNNs), inspired by the human brain, offer significant computational efficiency through discrete spike-based information transfer. Despite their potential to reduce inference energy consumption, a performance gap persists between SNNs and Artificial Neural Networks (ANNs), primarily due to current training methods and inherent model limitations. While recent research has aimed to enhance SNN learning by employing knowledge distillation (KD) from ANN teacher networks, traditional distillation techniques often overlook the distinctive spatiotemporal properties of SNNs, thus failing to fully leverage their advantages. To overcome these challenge, we propose a novel logit distillation method characterized by temporal separation and entropy regularization. This approach improves existing SNN distillation techniques by performing distillation learning on logits across different time steps, rather than merely on aggregated output features. Furthermore, the integration of entropy regularization stabilizes model optimization and further boosts the performance. Extensive experimental results indicate that our method surpasses prior SNN distillation strategies, whether based on logit distillation, feature distillation, or a combination of both. The code will be available on GitHub.","url_abs":"https://arxiv.org/abs/2503.03144v1","url_pdf":"https://arxiv.org/pdf/2503.03144v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"computational-efficiency","task_name":"Computational Efficiency"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"model-optimization","task_name":"Model Optimization"}],"methods":[{"method_slug":"entropy-regularization","method_name":"Entropy Regularization"},{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"},{"method_slug":"snn","method_name":"SNN"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2503.03144","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.03144"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/yukairong/TSER","reach":null}],"summary":{"ran":2,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4ee94b03e46c248e","entry":"Distiller","repo":"yukairong/TSER","repo_kind":"found_in_text","path":"model/distiller.py","file_url":"https://github.com/yukairong/TSER/blob/HEAD/model/distiller.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4ee94b03e46c248e"}},{"code_sha256_prefix":"4944f084b6f09509","entry":"FklLossWithReg","repo":"yukairong/TSER","repo_kind":"found_in_text","path":"model/distiller.py","file_url":"https://github.com/yukairong/TSER/blob/HEAD/model/distiller.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4944f084b6f09509"}},{"code_sha256_prefix":"a18395efbc1ce095","entry":"tet_loss","repo":"yukairong/TSER","repo_kind":"found_in_text","path":"model/distiller.py","file_url":"https://github.com/yukairong/TSER/blob/HEAD/model/distiller.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a18395efbc1ce095"}},{"code_sha256_prefix":"dd59e2a73bc626e5","entry":"KDT2","repo":"yukairong/TSER","repo_kind":"found_in_text","path":"model/distiller.py","file_url":"https://github.com/yukairong/TSER/blob/HEAD/model/distiller.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dd59e2a73bc626e5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}