{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2601-05707","title":"Multimodal In-context Learning for ASR of Low-resource Languages","arxiv_id":"2601.05707","date":"2026-01-09","proceeding":null,"authors":["Zhaolin Li","Jan Niehues"],"abstract":"Automatic speech recognition (ASR) still covers only a small fraction of the world's languages, mainly due to supervised data scarcity. In-context learning (ICL) with large language models (LLMs) addresses this problem, but prior work largely focuses on high-resource languages covered during training and text-only settings. This paper investigates whether speech LLMs can learn unseen languages with multimodal ICL (MICL), and how this learning can be used to improve ASR. We conduct experiments with two speech LLMs, Phi-4 and Qwen3-Omni, on three diverse endangered languages. Firstly, we find that MICL is effective for unseen languages, leveraging both speech and text modalities. We further show that cross-lingual transfer learning improves MICL efficiency on target languages without training on them. Moreover, we analyze attention patterns to interpret MICL mechanisms, and we observe layer-dependent preferences between audio and text context, with an overall bias towards text. Finally, we show that prompt-based ASR with speech LLMs performs poorly on unseen languages, motivating a simple ASR system that combines a stronger acoustic model with a speech LLM via MICL-based selection of acoustic hypotheses. Results show that MICL consistently improves ASR performance, and that cross-lingual transfer learning matches or outperforms corpus-trained language models without using target-language data. Our code is publicly available.","url_abs":"https://arxiv.org/abs/2601.05707","url_pdf":"https://arxiv.org/pdf/2601.05707","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2601.05707","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2601.05707"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/ZL-KA/MICL","reach":{"status":"ok"}}],"summary":{"unverified":9},"by_repo_kind":{"found_in_text":{"samples":9,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"00d83d2be5070834","entry":"cat_with_pad","repo":"ZL-KA/MICL","repo_kind":"found_in_text","path":"finetune_phi4.py","file_url":"https://github.com/ZL-KA/MICL/blob/HEAD/finetune_phi4.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"00d83d2be5070834"}},{"code_sha256_prefix":"42847b17a5c5d8ba","entry":"load_sonar_results","repo":"ZL-KA/MICL","repo_kind":"found_in_text","path":"ICL.py","file_url":"https://github.com/ZL-KA/MICL/blob/HEAD/ICL.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"42847b17a5c5d8ba"}},{"code_sha256_prefix":"6722ab5ede743d2f","entry":"map_preprocess_mlsuperb_hypo","repo":"ZL-KA/MICL","repo_kind":"found_in_text","path":"utils.py","file_url":"https://github.com/ZL-KA/MICL/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6722ab5ede743d2f"}},{"code_sha256_prefix":"34b0e8285f0c3635","entry":"ml_superb_text_normalization","repo":"ZL-KA/MICL","repo_kind":"found_in_text","path":"utils.py","file_url":"https://github.com/ZL-KA/MICL/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"34b0e8285f0c3635"}},{"code_sha256_prefix":"bde66d452167c327","entry":"normalize_standard","repo":"ZL-KA/MICL","repo_kind":"found_in_text","path":"evaluate.py","file_url":"https://github.com/ZL-KA/MICL/blob/HEAD/evaluate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bde66d452167c327"}},{"code_sha256_prefix":"5a7134ab177fe4f0","entry":"normalize_to_unit_range","repo":"ZL-KA/MICL","repo_kind":"found_in_text","path":"ICL.py","file_url":"https://github.com/ZL-KA/MICL/blob/HEAD/ICL.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5a7134ab177fe4f0"}},{"code_sha256_prefix":"2109d5ac2845731d","entry":"pad_sequence","repo":"ZL-KA/MICL","repo_kind":"found_in_text","path":"finetune_phi4.py","file_url":"https://github.com/ZL-KA/MICL/blob/HEAD/finetune_phi4.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2109d5ac2845731d"}},{"code_sha256_prefix":"e95ee381ed8fb2dd","entry":"remove_punctuation","repo":"ZL-KA/MICL","repo_kind":"found_in_text","path":"utils.py","file_url":"https://github.com/ZL-KA/MICL/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e95ee381ed8fb2dd"}},{"code_sha256_prefix":"b8fae5058b989c75","entry":"select_indices_for_icl","repo":"ZL-KA/MICL","repo_kind":"found_in_text","path":"ICL.py","file_url":"https://github.com/ZL-KA/MICL/blob/HEAD/ICL.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b8fae5058b989c75"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}