{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/xlstm-7b-a-recurrent-llm-for-fast-and","title":"xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference","arxiv_id":"2503.13427","date":"2025-03-17","proceeding":null,"authors":["Maximilian Beck","Korbinian Pöppel","Phillip Lippe","Richard Kurle","Patrick M. Blies","Günter Klambauer","Sebastian Böck","Sepp Hochreiter"],"abstract":"Recent breakthroughs in solving reasoning, math and coding problems with Large Language Models (LLMs) have been enabled by investing substantial computation budgets at inference time. Therefore, inference speed is one of the most critical properties of LLM architectures, and there is a growing need for LLMs that are efficient and fast at inference. Recently, LLMs built on the xLSTM architecture have emerged as a powerful alternative to Transformers, offering linear compute scaling with sequence length and constant memory usage, both highly desirable properties for efficient inference. However, such xLSTM-based LLMs have yet to be scaled to larger models and assessed and compared with respect to inference speed and efficiency. In this work, we introduce xLSTM 7B, a 7-billion-parameter LLM that combines xLSTM's architectural benefits with targeted optimizations for fast and efficient inference. Our experiments demonstrate that xLSTM 7B achieves performance on downstream tasks comparable to other similar-sized LLMs, while providing significantly faster inference speeds and greater efficiency compared to Llama- and Mamba-based LLMs. These results establish xLSTM 7B as the fastest and most efficient 7B LLM, offering a solution for tasks that require large amounts of test-time computation. Our work highlights xLSTM's potential as a foundational architecture for methods building on heavy use of LLM inference. Our model weights, model code and training code are open-source.","url_abs":"https://arxiv.org/abs/2503.13427v1","url_pdf":"https://arxiv.org/pdf/2503.13427v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"xlstm-7b-a-recurrent-llm-for-fast-and","repo_url":"https://github.com/nx-ai/xlstm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"xlstm-7b-a-recurrent-llm-for-fast-and","repo_url":"https://github.com/nx-ai/xlstm-jax","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"jax","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"xlstm-7b-a-recurrent-llm-for-fast-and","repo_url":"https://github.com/nx-ai/mlstm_kernels","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"mamba","task_name":"Mamba"},{"task_slug":"math","task_name":"Math"}],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2503.13427","atlas_url":"https://app.syntology.ai/?focus=2503.13427","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.13427"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nx-ai/xlstm","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nx-ai/xlstm-jax","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nx-ai/mlstm_kernels","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_draft_wrong":1,"ran":1,"unverified":3},"by_repo_kind":{"official":{"samples":5,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"705d17e5f5c0489b","entry":"bias_linspace_init_","repo":"nx-ai/xlstm","repo_kind":"official","path":"xlstm/components/init.py","file_url":"https://github.com/nx-ai/xlstm/blob/HEAD/xlstm/components/init.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"705d17e5f5c0489b"}},{"code_sha256_prefix":"2a6a1bfa026d5276","entry":"get_act_fn","repo":"nx-ai/xlstm","repo_kind":"official","path":"xlstm/components/feedforward.py","file_url":"https://github.com/nx-ai/xlstm/blob/HEAD/xlstm/components/feedforward.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2a6a1bfa026d5276"}},{"code_sha256_prefix":"25766cf2cb87920f","entry":"conv1d_step","repo":"nx-ai/xlstm","repo_kind":"official","path":"xlstm/components/conv.py","file_url":"https://github.com/nx-ai/xlstm/blob/HEAD/xlstm/components/conv.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"25766cf2cb87920f"}},{"code_sha256_prefix":"cfe16cc8552a2332","entry":"small_init_init_","repo":"nx-ai/xlstm","repo_kind":"official","path":"xlstm/components/init.py","file_url":"https://github.com/nx-ai/xlstm/blob/HEAD/xlstm/components/init.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cfe16cc8552a2332"}},{"code_sha256_prefix":"431c81ae48403967","entry":"wang_init_","repo":"nx-ai/xlstm","repo_kind":"official","path":"xlstm/components/init.py","file_url":"https://github.com/nx-ai/xlstm/blob/HEAD/xlstm/components/init.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"431c81ae48403967"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}