{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/state-free-inference-of-state-space-models","title":"State-Free Inference of State-Space Models: The Transfer Function Approach","arxiv_id":"2405.06147","date":"2024-05-10","proceeding":null,"authors":["Rom N. Parnichkun","Stefano Massaroli","Alessandro Moro","Jimmy T. H. Smith","Ramin Hasani","Mathias Lechner","Qi An","Christopher Ré","Hajime Asama","Stefano Ermon","Taiji Suzuki","Atsushi Yamashita","Michael Poli"],"abstract":"We approach designing a state-space model for deep learning applications through its dual representation, the transfer function, and uncover a highly efficient sequence parallel inference algorithm that is state-free: unlike other proposed algorithms, state-free inference does not incur any significant memory or computational cost with an increase in state size. We achieve this using properties of the proposed frequency domain transfer function parametrization, which enables direct computation of its corresponding convolutional kernel's spectrum via a single Fast Fourier Transform. Our experimental results across multiple sequence lengths and state sizes illustrates, on average, a 35% training speed improvement over S4 layers -- parametrized in time-domain -- on the Long Range Arena benchmark, while delivering state-of-the-art downstream performances over other attention-free approaches. Moreover, we report improved perplexity in language modeling over a long convolutional Hyena baseline, by simply introducing our transfer function parametrization. Our code is available at https://github.com/ruke1ire/RTF.","url_abs":"https://arxiv.org/abs/2405.06147v2","url_pdf":"https://arxiv.org/pdf/2405.06147v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"state-free-inference-of-state-space-models","repo_url":"https://github.com/ruke1ire/RTF","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"state-space-models","task_name":"State Space Models"}],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2405.06147","atlas_url":"https://app.syntology.ai/?focus=2405.06147","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.06147"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ruke1ire/RTF","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3051f8aeeb62bb52","entry":"montel_init","repo":"ruke1ire/RTF","repo_kind":"official","path":"rtf.py","file_url":"https://github.com/ruke1ire/RTF/blob/HEAD/rtf.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3051f8aeeb62bb52"}},{"code_sha256_prefix":"bb34448b589bbd0f","entry":"preprocess","repo":"ruke1ire/RTF","repo_kind":"official","path":"experiments/safari/evals/lambada.py","file_url":"https://github.com/ruke1ire/RTF/blob/HEAD/experiments/safari/evals/lambada.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bb34448b589bbd0f"}},{"code_sha256_prefix":"e8c14b7f4176d895","entry":"xavier_init","repo":"ruke1ire/RTF","repo_kind":"official","path":"rtf.py","file_url":"https://github.com/ruke1ire/RTF/blob/HEAD/rtf.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e8c14b7f4176d895"}},{"code_sha256_prefix":"1054e6a0f3337fc5","entry":"zeros_init","repo":"ruke1ire/RTF","repo_kind":"official","path":"rtf.py","file_url":"https://github.com/ruke1ire/RTF/blob/HEAD/rtf.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1054e6a0f3337fc5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}