{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2604-13073","title":"OmniTrace: A Unified Framework for Generation-Time Attribution in Omni-Modal LLMs","arxiv_id":"2604.13073","date":"2026-03-20","proceeding":null,"authors":["Qianqi Yan","Yichen Guo","Ching-Chen Kuo","Shan Jiang","Hang Yin","Yang Zhao","Xin Eric Wang"],"abstract":"Modern multimodal large language models (MLLMs) generate fluent responses from interleaved text, image, audio, and video inputs. However, identifying which input sources support each generated statement remains an open challenge. Existing attribution methods are primarily designed for classification settings, fixed prediction targets, or single-modality architectures, and do not naturally extend to autoregressive, decoder-only models performing open-ended multimodal generation. We introduce OmniTrace, a lightweight and model-agnostic framework that formalizes attribution as a generation-time tracing problem over the causal decoding process. OmniTrace provides a unified protocol that converts arbitrary token-level signals such as attention weights or gradient-based scores into coherent span-level, cross-modal explanations during decoding. It traces each generated token to multimodal inputs, aggregates signals into semantically meaningful spans, and selects concise supporting sources through confidence-weighted and temporally coherent aggregation, without retraining or supervision. Evaluations on Qwen2.5-Omni and MiniCPM-o-4.5 across visual, audio, and video tasks demonstrate that generation-aware span-level attribution produces more stable and interpretable explanations than naive self-attribution and embedding-based baselines, while remaining robust across multiple underlying attribution signals. Our results suggest that treating attribution as a structured generation-time tracing problem provides a scalable foundation for transparency in omni-modal language models.","url_abs":"https://arxiv.org/abs/2604.13073","url_pdf":"https://arxiv.org/pdf/2604.13073","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2604.13073","atlas_url":"https://app.syntology.ai/?focus=2604.13073","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2604.13073"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/openai/whisper","reach":null},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/modelscope/FunASR","reach":null}],"summary":{"ran":16,"ran_honours":2,"unverified":6},"by_repo_kind":{"found_in_text":{"samples":24,"ran":18,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e736efd2088f5330","entry":"add_file_root_path","repo":"modelscope/FunASR","repo_kind":"found_in_text","path":"funasr/download/download_model_from_hub.py","file_url":"https://github.com/modelscope/FunASR/blob/HEAD/funasr/download/download_model_from_hub.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e736efd2088f5330"}},{"code_sha256_prefix":"d94882bbd36f686d","entry":"check_vllm_applicable","repo":"modelscope/FunASR","repo_kind":"found_in_text","path":"funasr/auto/auto_model_vllm.py","file_url":"https://github.com/modelscope/FunASR/blob/HEAD/funasr/auto/auto_model_vllm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d94882bbd36f686d"}},{"code_sha256_prefix":"6c9da623e97fa4fe","entry":"cr_ctc_loss","repo":"modelscope/FunASR","repo_kind":"found_in_text","path":"funasr/losses/cr_ctc.py","file_url":"https://github.com/modelscope/FunASR/blob/HEAD/funasr/losses/cr_ctc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6c9da623e97fa4fe"}},{"code_sha256_prefix":"6e0808fc4828513a","entry":"exact_div","repo":"openai/whisper","repo_kind":"found_in_text","path":"whisper/utils.py","file_url":"https://github.com/openai/whisper/blob/HEAD/whisper/utils.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6e0808fc4828513a"}},{"code_sha256_prefix":"e40f7b6dc5c3977a","entry":"get_human_readable_count","repo":"modelscope/FunASR","repo_kind":"found_in_text","path":"funasr/train_utils/model_summary.py","file_url":"https://github.com/modelscope/FunASR/blob/HEAD/funasr/train_utils/model_summary.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e40f7b6dc5c3977a"}},{"code_sha256_prefix":"d6d4ea8b2806c520","entry":"install_requirements","repo":"modelscope/FunASR","repo_kind":"found_in_text","path":"funasr/utils/install_model_requirements.py","file_url":"https://github.com/modelscope/FunASR/blob/HEAD/funasr/utils/install_model_requirements.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d6d4ea8b2806c520"}},{"code_sha256_prefix":"6e70207ad77f0ae6","entry":"median_filter","repo":"openai/whisper","repo_kind":"found_in_text","path":"whisper/timing.py","file_url":"https://github.com/openai/whisper/blob/HEAD/whisper/timing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6e70207ad77f0ae6"}},{"code_sha256_prefix":"12c6c409a1e359c3","entry":"mel_filters","repo":"openai/whisper","repo_kind":"found_in_text","path":"whisper/audio.py","file_url":"https://github.com/openai/whisper/blob/HEAD/whisper/audio.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"12c6c409a1e359c3"}},{"code_sha256_prefix":"26200c7d0299db96","entry":"model_summary","repo":"modelscope/FunASR","repo_kind":"found_in_text","path":"funasr/train_utils/model_summary.py","file_url":"https://github.com/modelscope/FunASR/blob/HEAD/funasr/train_utils/model_summary.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"26200c7d0299db96"}},{"code_sha256_prefix":"e56996b20b9c19cb","entry":"optional_int","repo":"openai/whisper","repo_kind":"found_in_text","path":"whisper/utils.py","file_url":"https://github.com/openai/whisper/blob/HEAD/whisper/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e56996b20b9c19cb"}},{"code_sha256_prefix":"9f13de1c23e4003d","entry":"pad_or_trim","repo":"openai/whisper","repo_kind":"found_in_text","path":"whisper/audio.py","file_url":"https://github.com/openai/whisper/blob/HEAD/whisper/audio.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f13de1c23e4003d"}},{"code_sha256_prefix":"229df00957f1445e","entry":"pip_install_r","repo":"modelscope/FunASR","repo_kind":"found_in_text","path":"funasr/utils/install_model_requirements.py","file_url":"https://github.com/modelscope/FunASR/blob/HEAD/funasr/utils/install_model_requirements.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"229df00957f1445e"}},{"code_sha256_prefix":"fc2c2ae72cabe336","entry":"prepare_vllm_weights","repo":"modelscope/FunASR","repo_kind":"found_in_text","path":"funasr/auto/auto_model_vllm.py","file_url":"https://github.com/modelscope/FunASR/blob/HEAD/funasr/auto/auto_model_vllm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fc2c2ae72cabe336"}},{"code_sha256_prefix":"2206d84b6900ab3e","entry":"remove_symbols","repo":"openai/whisper","repo_kind":"found_in_text","path":"whisper/normalizers/basic.py","file_url":"https://github.com/openai/whisper/blob/HEAD/whisper/normalizers/basic.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2206d84b6900ab3e"}},{"code_sha256_prefix":"cef27f679a755971","entry":"remove_symbols_and_diacritics","repo":"openai/whisper","repo_kind":"found_in_text","path":"whisper/normalizers/basic.py","file_url":"https://github.com/openai/whisper/blob/HEAD/whisper/normalizers/basic.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cef27f679a755971"}},{"code_sha256_prefix":"e529c9641c178fe0","entry":"sinusoids","repo":"openai/whisper","repo_kind":"found_in_text","path":"whisper/model.py","file_url":"https://github.com/openai/whisper/blob/HEAD/whisper/model.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e529c9641c178fe0"}},{"code_sha256_prefix":"176b6ba6c2c2940c","entry":"str2bool","repo":"openai/whisper","repo_kind":"found_in_text","path":"whisper/utils.py","file_url":"https://github.com/openai/whisper/blob/HEAD/whisper/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"176b6ba6c2c2940c"}},{"code_sha256_prefix":"54138299a3e55e00","entry":"to_bytes","repo":"modelscope/FunASR","repo_kind":"found_in_text","path":"funasr/train_utils/model_summary.py","file_url":"https://github.com/modelscope/FunASR/blob/HEAD/funasr/train_utils/model_summary.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"54138299a3e55e00"}},{"code_sha256_prefix":"5288c864dbd0538d","entry":"get_or_download_model_dir","repo":"modelscope/FunASR","repo_kind":"found_in_text","path":"funasr/download/download_model_from_hub.py","file_url":"https://github.com/modelscope/FunASR/blob/HEAD/funasr/download/download_model_from_hub.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5288c864dbd0538d"}},{"code_sha256_prefix":"1c80497412c69732","entry":"get_or_download_model_dir_hf","repo":"modelscope/FunASR","repo_kind":"found_in_text","path":"funasr/download/download_model_from_hub.py","file_url":"https://github.com/modelscope/FunASR/blob/HEAD/funasr/download/download_model_from_hub.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1c80497412c69732"}},{"code_sha256_prefix":"80abc0b281d75575","entry":"load_audio","repo":"openai/whisper","repo_kind":"found_in_text","path":"whisper/audio.py","file_url":"https://github.com/openai/whisper/blob/HEAD/whisper/audio.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"80abc0b281d75575"}},{"code_sha256_prefix":"9c0b804792b26762","entry":"median_filter_cuda","repo":"openai/whisper","repo_kind":"found_in_text","path":"whisper/triton_ops.py","file_url":"https://github.com/openai/whisper/blob/HEAD/whisper/triton_ops.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9c0b804792b26762"}},{"code_sha256_prefix":"61d0c7cb5ccfdd5b","entry":"median_kernel","repo":"openai/whisper","repo_kind":"found_in_text","path":"whisper/triton_ops.py","file_url":"https://github.com/openai/whisper/blob/HEAD/whisper/triton_ops.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"61d0c7cb5ccfdd5b"}},{"code_sha256_prefix":"700202ff899b0c2d","entry":"transcribe","repo":"openai/whisper","repo_kind":"found_in_text","path":"whisper/transcribe.py","file_url":"https://github.com/openai/whisper/blob/HEAD/whisper/transcribe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"700202ff899b0c2d"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}