{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/finding-visual-saliency-in-continuous-spike","title":"Finding Visual Saliency in Continuous Spike Stream","arxiv_id":"2403.06233","date":"2024-03-10","proceeding":null,"authors":["Lin Zhu","Xianzhang Chen","Xiao Wang","Hua Huang"],"abstract":"As a bio-inspired vision sensor, the spike camera emulates the operational principles of the fovea, a compact retinal region, by employing spike discharges to encode the accumulation of per-pixel luminance intensity. Leveraging its high temporal resolution and bio-inspired neuromorphic design, the spike camera holds significant promise for advancing computer vision applications. Saliency detection mimics the behavior of human beings and captures the most salient region from the scenes. In this paper, we investigate the visual saliency in the continuous spike stream for the first time. To effectively process the binary spike stream, we propose a Recurrent Spiking Transformer (RST) framework, which is based on a full spiking neural network. Our framework enables the extraction of spatio-temporal features from the continuous spatio-temporal spike stream while maintaining low power consumption. To facilitate the training and validation of our proposed model, we build a comprehensive real-world spike-based visual saliency dataset, enriched with numerous light conditions. Extensive experiments demonstrate the superior performance of our Recurrent Spiking Transformer framework in comparison to other spike neural network-based methods. Our framework exhibits a substantial margin of improvement in capturing and highlighting visual saliency in the spike stream, which not only provides a new perspective for spike-based saliency segmentation but also shows a new paradigm for full SNN-based transformer models. The code and dataset are available at \\url{https://github.com/BIT-Vision/SVS}.","url_abs":"https://arxiv.org/abs/2403.06233v1","url_pdf":"https://arxiv.org/pdf/2403.06233v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"finding-visual-saliency-in-continuous-spike","repo_url":"https://github.com/bit-vision/svs","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"saliency-detection","task_name":"Saliency Detection"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.06233","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.06233"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/BIT-Vision/SVS","reach":{"status":"ok"}}],"summary":{"ran":3,"ran_honours":2,"unverified":6},"by_repo_kind":{"official":{"samples":11,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"090f923270153638","entry":"Object","repo":"BIT-Vision/SVS","repo_kind":"official","path":"utils/metric.py","file_url":"https://github.com/BIT-Vision/SVS/blob/HEAD/utils/metric.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"090f923270153638"}},{"code_sha256_prefix":"cffa82b90cff4af2","entry":"S_object","repo":"BIT-Vision/SVS","repo_kind":"official","path":"utils/metric.py","file_url":"https://github.com/BIT-Vision/SVS/blob/HEAD/utils/metric.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cffa82b90cff4af2"}},{"code_sha256_prefix":"6154e3744ece5728","entry":"create_window","repo":"BIT-Vision/SVS","repo_kind":"official","path":"utils/pytorch_ssim.py","file_url":"https://github.com/BIT-Vision/SVS/blob/HEAD/utils/pytorch_ssim.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6154e3744ece5728"}},{"code_sha256_prefix":"a705368da8e00322","entry":"decode_compress_spike_from_numpy","repo":"BIT-Vision/SVS","repo_kind":"official","path":"datasets/dataset.py","file_url":"https://github.com/BIT-Vision/SVS/blob/HEAD/datasets/dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a705368da8e00322"}},{"code_sha256_prefix":"c56b7ef16f309a45","entry":"gaussian","repo":"BIT-Vision/SVS","repo_kind":"official","path":"utils/pytorch_ssim.py","file_url":"https://github.com/BIT-Vision/SVS/blob/HEAD/utils/pytorch_ssim.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c56b7ef16f309a45"}},{"code_sha256_prefix":"dc0d8296a6b48006","entry":"StructureMeasure","repo":"BIT-Vision/SVS","repo_kind":"official","path":"utils/metric.py","file_url":"https://github.com/BIT-Vision/SVS/blob/HEAD/utils/metric.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dc0d8296a6b48006"}},{"code_sha256_prefix":"a82c9310f46b74cb","entry":"get_datasets","repo":"BIT-Vision/SVS","repo_kind":"official","path":"datasets/dataset.py","file_url":"https://github.com/BIT-Vision/SVS/blob/HEAD/datasets/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a82c9310f46b74cb"}},{"code_sha256_prefix":"7424d53e0ec86e0a","entry":"get_transforms","repo":"BIT-Vision/SVS","repo_kind":"official","path":"datasets/transforms.py","file_url":"https://github.com/BIT-Vision/SVS/blob/HEAD/datasets/transforms.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7424d53e0ec86e0a"}},{"code_sha256_prefix":"998ee6fe224c2522","entry":"load_checkpoint","repo":"BIT-Vision/SVS","repo_kind":"official","path":"models/load_model.py","file_url":"https://github.com/BIT-Vision/SVS/blob/HEAD/models/load_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"998ee6fe224c2522"}},{"code_sha256_prefix":"a56c89e70402fe2f","entry":"load_model","repo":"BIT-Vision/SVS","repo_kind":"official","path":"models/load_model.py","file_url":"https://github.com/BIT-Vision/SVS/blob/HEAD/models/load_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a56c89e70402fe2f"}},{"code_sha256_prefix":"9151c793f90473f0","entry":"ssim","repo":"BIT-Vision/SVS","repo_kind":"official","path":"utils/pytorch_ssim.py","file_url":"https://github.com/BIT-Vision/SVS/blob/HEAD/utils/pytorch_ssim.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9151c793f90473f0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}