{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/provable-benefits-of-complex","title":"Provable Benefits of Complex Parameterizations for Structured State Space Models","arxiv_id":"2410.14067","date":"2024-10-17","proceeding":null,"authors":["Yuval Ran-Milo","Eden Lumbroso","Edo Cohen-Karlik","Raja Giryes","Amir Globerson","Nadav Cohen"],"abstract":"Structured state space models (SSMs), the core engine behind prominent neural networks such as S4 and Mamba, are linear dynamical systems adhering to a specified structure, most notably diagonal. In contrast to typical neural network modules, whose parameterizations are real, SSMs often use complex parameterizations. Theoretically explaining the benefits of complex parameterizations for SSMs is an open problem. The current paper takes a step towards its resolution, by establishing formal gaps between real and complex diagonal SSMs. Firstly, we prove that while a moderate dimension suffices in order for a complex SSM to express all mappings of a real SSM, a much higher dimension is needed for a real SSM to express mappings of a complex SSM. Secondly, we prove that even if the dimension of a real SSM is high enough to express a given mapping, typically, doing so requires the parameters of the real SSM to hold exponentially large values, which cannot be learned in practice. In contrast, a complex SSM can express any given mapping with moderate parameter values. Experiments corroborate our theory, and suggest a potential extension of the theory that accounts for selectivity, a new architectural feature yielding state of the art performance.","url_abs":"https://arxiv.org/abs/2410.14067v2","url_pdf":"https://arxiv.org/pdf/2410.14067v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"provable-benefits-of-complex","repo_url":"https://github.com/edenlum/ssmcomplexparambenefits","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"mamba","task_name":"Mamba"},{"task_slug":"state-space-models","task_name":"State Space Models"}],"methods":[{"method_slug":"mamba","method_name":"Mamba"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2410.14067","atlas_url":"https://app.syntology.ai/?focus=2410.14067","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.14067"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/state-spaces/s4","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/alxndrTL/mamba.py","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/edenlum/ssmcomplexparambenefits","reach":{"status":"ok"}}],"summary":{"ran_fixture":1,"ran":11,"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":11,"ran":10,"repositories":1},"found_in_text":{"samples":6,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"3c76e52815c5401d","entry":"repeat_kv","repo":"alxndrTL/mamba.py","repo_kind":"found_in_text","path":"mambapy/jamba.py","file_url":"https://github.com/alxndrTL/mamba.py/blob/HEAD/mambapy/jamba.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":2,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3c76e52815c5401d"}},{"code_sha256_prefix":"83d17705e31bdff1","entry":"cauchy_naive","repo":"edenlum/ssmcomplexparambenefits","repo_kind":"official","path":"s4.py","file_url":"https://github.com/edenlum/ssmcomplexparambenefits/blob/HEAD/s4.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"83d17705e31bdff1"}},{"code_sha256_prefix":"9828d3d0e675c964","entry":"complex_log","repo":"edenlum/ssmcomplexparambenefits","repo_kind":"official","path":"simple_mamba/pscan.py","file_url":"https://github.com/edenlum/ssmcomplexparambenefits/blob/HEAD/simple_mamba/pscan.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9828d3d0e675c964"}},{"code_sha256_prefix":"24adb09177909a7d","entry":"convert_type","repo":"edenlum/ssmcomplexparambenefits","repo_kind":"official","path":"utils.py","file_url":"https://github.com/edenlum/ssmcomplexparambenefits/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"24adb09177909a7d"}},{"code_sha256_prefix":"9a722661e545209d","entry":"delay_l2","repo":"edenlum/ssmcomplexparambenefits","repo_kind":"official","path":"datasets.py","file_url":"https://github.com/edenlum/ssmcomplexparambenefits/blob/HEAD/datasets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9a722661e545209d"}},{"code_sha256_prefix":"999d44c96aa5e190","entry":"get_hash","repo":"edenlum/ssmcomplexparambenefits","repo_kind":"official","path":"utils.py","file_url":"https://github.com/edenlum/ssmcomplexparambenefits/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"999d44c96aa5e190"}},{"code_sha256_prefix":"61139ec62260b411","entry":"get_logger","repo":"edenlum/ssmcomplexparambenefits","repo_kind":"official","path":"s4.py","file_url":"https://github.com/edenlum/ssmcomplexparambenefits/blob/HEAD/s4.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"61139ec62260b411"}},{"code_sha256_prefix":"85c040cde89f147b","entry":"get_target_imp_res","repo":"edenlum/ssmcomplexparambenefits","repo_kind":"official","path":"theory_ssm/theory_ssm.py","file_url":"https://github.com/edenlum/ssmcomplexparambenefits/blob/HEAD/theory_ssm/theory_ssm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"85c040cde89f147b"}},{"code_sha256_prefix":"bbc00961f58e024d","entry":"impulse_response","repo":"edenlum/ssmcomplexparambenefits","repo_kind":"official","path":"theory_ssm/theory_ssm.py","file_url":"https://github.com/edenlum/ssmcomplexparambenefits/blob/HEAD/theory_ssm/theory_ssm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bbc00961f58e024d"}},{"code_sha256_prefix":"0ebccfcf2c0a35d9","entry":"load_balancing_loss","repo":"alxndrTL/mamba.py","repo_kind":"found_in_text","path":"mambapy/jamba.py","file_url":"https://github.com/alxndrTL/mamba.py/blob/HEAD/mambapy/jamba.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0ebccfcf2c0a35d9"}},{"code_sha256_prefix":"264ebea7a7690cc2","entry":"log_vandermonde_naive","repo":"edenlum/ssmcomplexparambenefits","repo_kind":"official","path":"s4.py","file_url":"https://github.com/edenlum/ssmcomplexparambenefits/blob/HEAD/s4.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"264ebea7a7690cc2"}},{"code_sha256_prefix":"23c18f489a46fdc1","entry":"override_config","repo":"edenlum/ssmcomplexparambenefits","repo_kind":"official","path":"utils.py","file_url":"https://github.com/edenlum/ssmcomplexparambenefits/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"23c18f489a46fdc1"}},{"code_sha256_prefix":"152101b1129338c6","entry":"unsqueeze","repo":"alxndrTL/mamba.py","repo_kind":"found_in_text","path":"mlx/misc.py","file_url":"https://github.com/alxndrTL/mamba.py/blob/HEAD/mlx/misc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"152101b1129338c6"}},{"code_sha256_prefix":"6da707934cf98642","entry":"checkpointed_sequence","repo":"alxndrTL/mamba.py","repo_kind":"found_in_text","path":"mambapy/checkpointing.py","file_url":"https://github.com/alxndrTL/mamba.py/blob/HEAD/mambapy/checkpointing.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6da707934cf98642"}},{"code_sha256_prefix":"0ffc6db74abcbaca","entry":"logits_to_probability","repo":"edenlum/ssmcomplexparambenefits","repo_kind":"official","path":"datasets.py","file_url":"https://github.com/edenlum/ssmcomplexparambenefits/blob/HEAD/datasets.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0ffc6db74abcbaca"}},{"code_sha256_prefix":"d547c23b7746ffe5","entry":"npo2","repo":"alxndrTL/mamba.py","repo_kind":"found_in_text","path":"mambapy/pscan.py","file_url":"https://github.com/alxndrTL/mamba.py/blob/HEAD/mambapy/pscan.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d547c23b7746ffe5"}},{"code_sha256_prefix":"8213e85fd713879a","entry":"pad_npo2","repo":"alxndrTL/mamba.py","repo_kind":"found_in_text","path":"mambapy/pscan.py","file_url":"https://github.com/alxndrTL/mamba.py/blob/HEAD/mambapy/pscan.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8213e85fd713879a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}