{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/simplified-state-space-layers-for-sequence","title":"Simplified State Space Layers for Sequence Modeling","arxiv_id":"2208.04933","date":"2022-08-09","proceeding":null,"authors":["Jimmy T. H. Smith","Andrew Warrington","Scott W. Linderman"],"abstract":"Models using structured state space sequence (S4) layers have achieved state-of-the-art performance on long-range sequence modeling tasks. An S4 layer combines linear state space models (SSMs), the HiPPO framework, and deep learning to achieve high performance. We build on the design of the S4 layer and introduce a new state space layer, the S5 layer. Whereas an S4 layer uses many independent single-input, single-output SSMs, the S5 layer uses one multi-input, multi-output SSM. We establish a connection between S5 and S4, and use this to develop the initialization and parameterization used by the S5 model. The result is a state space layer that can leverage efficient and widely implemented parallel scans, allowing S5 to match the computational efficiency of S4, while also achieving state-of-the-art performance on several long-range sequence modeling tasks. S5 averages 87.4% on the long range arena benchmark, and 98.5% on the most difficult Path-X task.","url_abs":"https://arxiv.org/abs/2208.04933v3","url_pdf":"https://arxiv.org/pdf/2208.04933v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"simplified-state-space-layers-for-sequence","repo_url":"https://github.com/lindermanlab/S5","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"simplified-state-space-layers-for-sequence","repo_url":"https://github.com/Efficient-Scalable-Machine-Learning/event-ssm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"simplified-state-space-layers-for-sequence","repo_url":"https://github.com/leonty1/deepldnn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"simplified-state-space-layers-for-sequence","repo_url":"https://github.com/leonty1/essm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"simplified-state-space-layers-for-sequence","repo_url":"https://github.com/msgwak/last","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok"}},{"paper_slug":"simplified-state-space-layers-for-sequence","repo_url":"https://github.com/uzh-rpg/ssms_event_cameras","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"jax","reach":{"status":"ok"}}],"tasks":[{"task_slug":"computational-efficiency","task_name":"Computational Efficiency"},{"task_slug":"listops","task_name":"ListOps"},{"task_slug":"long-range-modeling","task_name":"Long-range modeling"},{"task_slug":null,"task_name":"Pathfinder"},{"task_slug":null,"task_name":"Pathfinder-X"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"state-space-models","task_name":"State Space Models"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/long-range-modeling-on-lra","task":"Long-range modeling","dataset":"LRA","model":"S5","rank_in_archive_order":1,"of":7,"metrics":{"Avg":"87.46","Image":"88","ListOps":"62.15","Pathfinder":"95.33","Pathfinder-X":"98.58","Retrieval":"91.4","Text":"89.31"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2208.04933","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2208.04933"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lindermanlab/S5","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/msgwak/last","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/leonty1/deepldnn","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Efficient-Scalable-Machine-Learning/event-ssm","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/leonty1/essm","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/uzh-rpg/ssms_event_cameras","reach":{"status":"ok"}}],"summary":{"ran_fixture":1,"ran":3,"unverified":15},"by_repo_kind":{"listed":{"samples":16,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"b4fc72406d2ff673","entry":"binary_operator","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"b4fc72406d2ff673"}},{"code_sha256_prefix":"ce310b19a858070f","entry":"dequantize","repo":"leonty1/deepldnn","repo_kind":"listed","path":"src/dataloaders/audio.py","file_url":"https://github.com/leonty1/deepldnn/blob/HEAD/src/dataloaders/audio.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ce310b19a858070f"}},{"code_sha256_prefix":"1b76e4737067023f","entry":"minmax_scale","repo":"leonty1/deepldnn","repo_kind":"listed","path":"src/dataloaders/audio.py","file_url":"https://github.com/leonty1/deepldnn/blob/HEAD/src/dataloaders/audio.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1b76e4737067023f"}},{"code_sha256_prefix":"1631cd936e9b1a90","entry":"quantize","repo":"leonty1/deepldnn","repo_kind":"listed","path":"src/dataloaders/audio.py","file_url":"https://github.com/leonty1/deepldnn/blob/HEAD/src/dataloaders/audio.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1631cd936e9b1a90"}},{"code_sha256_prefix":"25c8f92b07a28373","entry":"adding_static_dataset","repo":"leonty1/deepldnn","repo_kind":"listed","path":"src/dataloaders/adding.py","file_url":"https://github.com/leonty1/deepldnn/blob/HEAD/src/dataloaders/adding.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"25c8f92b07a28373"}},{"code_sha256_prefix":"5db05fd542b84048","entry":"copying_static_dataset","repo":"leonty1/deepldnn","repo_kind":"listed","path":"src/dataloaders/copying.py","file_url":"https://github.com/leonty1/deepldnn/blob/HEAD/src/dataloaders/copying.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5db05fd542b84048"}},{"code_sha256_prefix":"c0135e087b049ad1","entry":"discretize_bilinear","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"c0135e087b049ad1"}},{"code_sha256_prefix":"f9485e8e0d1e9aee","entry":"discretize_zoh","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"f9485e8e0d1e9aee"}},{"code_sha256_prefix":"72d464924b5dee96","entry":"listops_tokenizer","repo":"leonty1/deepldnn","repo_kind":"listed","path":"src/dataloaders/datasets.py","file_url":"https://github.com/leonty1/deepldnn/blob/HEAD/src/dataloaders/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"72d464924b5dee96"}},{"code_sha256_prefix":"1bdd2b1087ff31b1","entry":"load_data","repo":"leonty1/deepldnn","repo_kind":"listed","path":"src/dataloaders/sc.py","file_url":"https://github.com/leonty1/deepldnn/blob/HEAD/src/dataloaders/sc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1bdd2b1087ff31b1"}},{"code_sha256_prefix":"b9da9054915aabbf","entry":"np_copying_data","repo":"leonty1/deepldnn","repo_kind":"listed","path":"src/dataloaders/copying.py","file_url":"https://github.com/leonty1/deepldnn/blob/HEAD/src/dataloaders/copying.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b9da9054915aabbf"}},{"code_sha256_prefix":"a06af5ba6942100d","entry":"pad","repo":"leonty1/deepldnn","repo_kind":"listed","path":"src/dataloaders/sc.py","file_url":"https://github.com/leonty1/deepldnn/blob/HEAD/src/dataloaders/sc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a06af5ba6942100d"}},{"code_sha256_prefix":"037cf309e950d7ef","entry":"subsample","repo":"leonty1/deepldnn","repo_kind":"listed","path":"src/dataloaders/sc.py","file_url":"https://github.com/leonty1/deepldnn/blob/HEAD/src/dataloaders/sc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"037cf309e950d7ef"}},{"code_sha256_prefix":"9d0777602eda354e","entry":"time_features","repo":"leonty1/deepldnn","repo_kind":"listed","path":"src/dataloaders/et.py","file_url":"https://github.com/leonty1/deepldnn/blob/HEAD/src/dataloaders/et.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9d0777602eda354e"}},{"code_sha256_prefix":"137297b83ef054cd","entry":"time_features","repo":"leonty1/deepldnn","repo_kind":"listed","path":"src/dataloaders/stocks.py","file_url":"https://github.com/leonty1/deepldnn/blob/HEAD/src/dataloaders/stocks.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"137297b83ef054cd"}},{"code_sha256_prefix":"f8544563682146e5","entry":"time_features_from_frequency_str","repo":"leonty1/deepldnn","repo_kind":"listed","path":"src/dataloaders/et.py","file_url":"https://github.com/leonty1/deepldnn/blob/HEAD/src/dataloaders/et.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f8544563682146e5"}},{"code_sha256_prefix":"6f2f58700f9f2450","entry":"tokenize_raw","repo":"leonty1/deepldnn","repo_kind":"listed","path":"src/dataloaders/lm.py","file_url":"https://github.com/leonty1/deepldnn/blob/HEAD/src/dataloaders/lm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6f2f58700f9f2450"}},{"code_sha256_prefix":"6af6c9fb197b8e6b","entry":"torch_adding_data","repo":"leonty1/deepldnn","repo_kind":"listed","path":"src/dataloaders/adding.py","file_url":"https://github.com/leonty1/deepldnn/blob/HEAD/src/dataloaders/adding.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6af6c9fb197b8e6b"}},{"code_sha256_prefix":"3d3875d2fc5f4975","entry":"torch_copying_data","repo":"leonty1/deepldnn","repo_kind":"listed","path":"src/dataloaders/copying.py","file_url":"https://github.com/leonty1/deepldnn/blob/HEAD/src/dataloaders/copying.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3d3875d2fc5f4975"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}