{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/state-space-models-for-event-cameras","title":"State Space Models for Event Cameras","arxiv_id":"2402.15584","date":"2024-02-23","proceeding":"CVPR 2024 1","authors":["Nikola Zubić","Mathias Gehrig","Davide Scaramuzza"],"abstract":"Today, state-of-the-art deep neural networks that process event-camera data first convert a temporal window of events into dense, grid-like input representations. As such, they exhibit poor generalizability when deployed at higher inference frequencies (i.e., smaller temporal windows) than the ones they were trained on. We address this challenge by introducing state-space models (SSMs) with learnable timescale parameters to event-based vision. This design adapts to varying frequencies without the need to retrain the network at different frequencies. Additionally, we investigate two strategies to counteract aliasing effects when deploying the model at higher frequencies. We comprehensively evaluate our approach against existing methods based on RNN and Transformer architectures across various benchmarks, including Gen1 and 1 Mpx event camera datasets. Our results demonstrate that SSM-based models train 33% faster and also exhibit minimal performance degradation when tested at higher frequencies than the training input. Traditional RNN and Transformer models exhibit performance drops of more than 20 mAP, with SSMs having a drop of 3.76 mAP, highlighting the effectiveness of SSMs in event-based vision tasks.","url_abs":"https://arxiv.org/abs/2402.15584v3","url_pdf":"https://arxiv.org/pdf/2402.15584v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"state-space-models-for-event-cameras","repo_url":"https://github.com/uzh-rpg/ssms_event_cameras","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok"}},{"paper_slug":"state-space-models-for-event-cameras","repo_url":"https://github.com/lindermanlab/S5","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"event-based-vision","task_name":"Event-based vision"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"state-space-models","task_name":"State Space Models"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/object-detection-on-gen1-detection","task":"Object Detection","dataset":"GEN1 Detection","model":"S5-ViT-B","rank_in_archive_order":3,"of":11,"metrics":{"Params":"18.2","mAP":"47.4"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-gen1-detection","task":"Object Detection","dataset":"GEN1 Detection","model":"S4D-ViT-B","rank_in_archive_order":7,"of":11,"metrics":{"Params":"16.5","mAP":"46.2"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.15584","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.15584"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lindermanlab/S5","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/uzh-rpg/ssms_event_cameras","reach":{"status":"ok"}}],"summary":{"ran_fixture":1,"ran":4,"unverified":14},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1},"listed":{"samples":18,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"b4fc72406d2ff673","entry":"binary_operator","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/ssm.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/ssm.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b4fc72406d2ff673"}},{"code_sha256_prefix":"ce310b19a858070f","entry":"dequantize","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/dataloaders/audio.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/dataloaders/audio.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ce310b19a858070f"}},{"code_sha256_prefix":"11319553be995a86","entry":"get_ckpt_callback","repo":"uzh-rpg/ssms_event_cameras","repo_kind":"official","path":"RVT/callbacks/custom.py","file_url":"https://github.com/uzh-rpg/ssms_event_cameras/blob/HEAD/RVT/callbacks/custom.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"11319553be995a86"}},{"code_sha256_prefix":"1b76e4737067023f","entry":"minmax_scale","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/dataloaders/audio.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/dataloaders/audio.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1b76e4737067023f"}},{"code_sha256_prefix":"1631cd936e9b1a90","entry":"quantize","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/dataloaders/audio.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/dataloaders/audio.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1631cd936e9b1a90"}},{"code_sha256_prefix":"faeebb869aee2124","entry":"cosine_annealing","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/train_helpers.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/train_helpers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"faeebb869aee2124"}},{"code_sha256_prefix":"476b5f7f116d7c9b","entry":"create_lra_imdb_classification_dataset","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/dataloading.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/dataloading.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"476b5f7f116d7c9b"}},{"code_sha256_prefix":"52bd0069ea925687","entry":"create_lra_listops_classification_dataset","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/dataloading.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/dataloading.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"52bd0069ea925687"}},{"code_sha256_prefix":"25edc240223b76c0","entry":"deprecated","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/dataloaders/base.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/dataloaders/base.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"25edc240223b76c0"}},{"code_sha256_prefix":"c0135e087b049ad1","entry":"discretize_bilinear","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/ssm.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/ssm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c0135e087b049ad1"}},{"code_sha256_prefix":"f9485e8e0d1e9aee","entry":"discretize_zoh","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/ssm.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/ssm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9485e8e0d1e9aee"}},{"code_sha256_prefix":"a2cd676d4d4c4910","entry":"linear_warmup","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/train_helpers.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/train_helpers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a2cd676d4d4c4910"}},{"code_sha256_prefix":"72d464924b5dee96","entry":"listops_tokenizer","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/dataloaders/lra.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/dataloaders/lra.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"72d464924b5dee96"}},{"code_sha256_prefix":"2b5694fa1220f8bd","entry":"make_DPLR_HiPPO","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/ssm_init.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/ssm_init.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b5694fa1220f8bd"}},{"code_sha256_prefix":"3f26e3f7ea0487f3","entry":"make_HiPPO","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/ssm_init.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/ssm_init.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3f26e3f7ea0487f3"}},{"code_sha256_prefix":"0b867bda221aedfc","entry":"make_NPLR_HiPPO","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/ssm_init.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/ssm_init.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0b867bda221aedfc"}},{"code_sha256_prefix":"508a42e36707cd15","entry":"make_data_loader","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/dataloading.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/dataloading.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"508a42e36707cd15"}},{"code_sha256_prefix":"261d3ddc455af2a0","entry":"masked_meanpool","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/seq_model.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/seq_model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"261d3ddc455af2a0"}},{"code_sha256_prefix":"2802de0d2519b69a","entry":"reduce_lr_on_plateau","repo":"lindermanlab/S5","repo_kind":"listed","path":"s5/train_helpers.py","file_url":"https://github.com/lindermanlab/S5/blob/HEAD/s5/train_helpers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2802de0d2519b69a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}