{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2604-12928","title":"MoshiRAG: Asynchronous Knowledge Retrieval for Full-Duplex Speech Language Models","arxiv_id":"2604.12928","date":"2026-04-14","proceeding":null,"authors":["Chung-Ming Chien","Manu Orsini","Eugene Kharitonov","Neil Zeghidour","Karen Livescu","Alexandre Défossez"],"abstract":"Speech-to-speech language models have recently emerged to enhance the naturalness of conversational AI. In particular, full-duplex models are distinguished by their real-time interactivity, including handling of pauses, interruptions, and backchannels. However, improving their factuality remains an open challenge. While scaling the model size could address this gap, it would make real-time inference prohibitively expensive. In this work, we propose MoshiRAG, a modular approach that combines a compact full-duplex interface with selective retrieval to access more powerful knowledge sources. Our asynchronous framework enables the model to identify knowledge-demanding queries and ground its responses in external information. By leveraging the natural temporal gap between response onset and the delivery of core information, the retrieval process can be completed while maintaining a natural conversation flow. With this approach, MoshiRAG achieves factuality comparable to the best publicly released non-duplex speech language models while preserving the interactivity inherent to full-duplex systems. Moreover, our flexible design supports plug-and-play retrieval methods without retraining and demonstrates strong performance on out-of-domain mathematical reasoning tasks.","url_abs":"https://arxiv.org/abs/2604.12928","url_pdf":"https://arxiv.org/pdf/2604.12928","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2604.12928","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2604.12928"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/kyutai-labs/moshi-rag","reach":null}],"summary":{"unverified":6},"by_repo_kind":{"found_in_text":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"61632108e112746f","entry":"apply_parametrization_norm","repo":"kyutai-labs/moshi-rag","repo_kind":"found_in_text","path":"moshi/moshi/modules/conv.py","file_url":"https://github.com/kyutai-labs/moshi-rag/blob/HEAD/moshi/moshi/modules/conv.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"61632108e112746f"}},{"code_sha256_prefix":"dfeda1e8168320ae","entry":"extract_answer_variants","repo":"kyutai-labs/moshi-rag","repo_kind":"found_in_text","path":"moshi/moshi/evaluate/score.py","file_url":"https://github.com/kyutai-labs/moshi-rag/blob/HEAD/moshi/moshi/evaluate/score.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dfeda1e8168320ae"}},{"code_sha256_prefix":"26ecc97e6c0629ee","entry":"gating_forward_generic","repo":"kyutai-labs/moshi-rag","repo_kind":"found_in_text","path":"moshi/moshi/modules/gating.py","file_url":"https://github.com/kyutai-labs/moshi-rag/blob/HEAD/moshi/moshi/modules/gating.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"26ecc97e6c0629ee"}},{"code_sha256_prefix":"5a9465546cfaa0f0","entry":"get_extra_padding_for_conv1d","repo":"kyutai-labs/moshi-rag","repo_kind":"found_in_text","path":"moshi/moshi/modules/conv.py","file_url":"https://github.com/kyutai-labs/moshi-rag/blob/HEAD/moshi/moshi/modules/conv.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5a9465546cfaa0f0"}},{"code_sha256_prefix":"625b33a3b1ad0220","entry":"hf_get","repo":"kyutai-labs/moshi-rag","repo_kind":"found_in_text","path":"moshi/moshi/models/loaders.py","file_url":"https://github.com/kyutai-labs/moshi-rag/blob/HEAD/moshi/moshi/models/loaders.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"625b33a3b1ad0220"}},{"code_sha256_prefix":"44c94244e949b9fb","entry":"pad_for_conv1d","repo":"kyutai-labs/moshi-rag","repo_kind":"found_in_text","path":"moshi/moshi/modules/conv.py","file_url":"https://github.com/kyutai-labs/moshi-rag/blob/HEAD/moshi/moshi/modules/conv.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"44c94244e949b9fb"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}