{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/accelerating-toeplitz-neural-network-with","title":"Accelerating Toeplitz Neural Network with Constant-time Inference Complexity","arxiv_id":"2311.08756","date":"2023-11-15","proceeding":null,"authors":["Zhen Qin","Yiran Zhong"],"abstract":"Toeplitz Neural Networks (TNNs) have exhibited outstanding performance in various sequence modeling tasks. They outperform commonly used Transformer-based models while benefiting from log-linear space-time complexities. On the other hand, State Space Models (SSMs) achieve lower performance than TNNs in language modeling but offer the advantage of constant inference complexity. In this paper, we aim to combine the strengths of TNNs and SSMs by converting TNNs to SSMs during inference, thereby enabling TNNs to achieve the same constant inference complexities as SSMs. To accomplish this, we formulate the conversion process as an optimization problem and provide a closed-form solution. We demonstrate how to transform the target equation into a Vandermonde linear system problem, which can be efficiently solved using the Discrete Fourier Transform (DFT). Notably, our method requires no training and maintains numerical stability. It can be also applied to any LongConv-based model. To assess its effectiveness, we conduct extensive experiments on language modeling tasks across various settings. Additionally, we compare our method to other gradient-descent solutions, highlighting the superior numerical stability of our approach. The source code is available at https://github.com/OpenNLPLab/ETSC-Exact-Toeplitz-to-SSM-Conversion.","url_abs":"https://arxiv.org/abs/2311.08756v1","url_pdf":"https://arxiv.org/pdf/2311.08756v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"accelerating-toeplitz-neural-network-with","repo_url":"https://github.com/opennlplab/etsc-exact-toeplitz-to-ssm-conversion","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"state-space-models","task_name":"State Space Models"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2311.08756","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.08756"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/opennlplab/etsc-exact-toeplitz-to-ssm-conversion","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/OpenNLPLab/ETSC-Exact-Toeplitz-to-SSM-Conversion","reach":{"status":"ok"}}],"summary":{"ran":2,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"e587d8232ac158a1","entry":"get_mem_space","repo":"opennlplab/etsc-exact-toeplitz-to-ssm-conversion","repo_kind":"official","path":"gpu_mem_track.py","file_url":"https://github.com/opennlplab/etsc-exact-toeplitz-to-ssm-conversion/blob/HEAD/gpu_mem_track.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e587d8232ac158a1"}},{"code_sha256_prefix":"9ad1c10cb9cb3291","entry":"transformation","repo":"OpenNLPLab/ETSC-Exact-Toeplitz-to-SSM-Conversion","repo_kind":"official","path":"transformation.py","file_url":"https://github.com/OpenNLPLab/ETSC-Exact-Toeplitz-to-SSM-Conversion/blob/HEAD/transformation.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9ad1c10cb9cb3291"}},{"code_sha256_prefix":"a35ddce7a07cbe75","entry":"transformation","repo":"opennlplab/etsc-exact-toeplitz-to-ssm-conversion","repo_kind":"official","path":"transformation.py","file_url":"https://github.com/opennlplab/etsc-exact-toeplitz-to-ssm-conversion/blob/HEAD/transformation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a35ddce7a07cbe75"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}