{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-robust-multimodal-sentiment-analysis","title":"Towards Robust Multimodal Sentiment Analysis with Incomplete Data","arxiv_id":"2409.20012","date":"2024-09-30","proceeding":null,"authors":["Haoyu Zhang","Wenbin Wang","Tianshu Yu"],"abstract":"The field of Multimodal Sentiment Analysis (MSA) has recently witnessed an emerging direction seeking to tackle the issue of data incompleteness. Recognizing that the language modality typically contains dense sentiment information, we consider it as the dominant modality and present an innovative Language-dominated Noise-resistant Learning Network (LNLN) to achieve robust MSA. The proposed LNLN features a dominant modality correction (DMC) module and dominant modality based multimodal learning (DMML) module, which enhances the model's robustness across various noise scenarios by ensuring the quality of dominant modality representations. Aside from the methodical design, we perform comprehensive experiments under random data missing scenarios, utilizing diverse and meaningful settings on several popular datasets (\\textit{e.g.,} MOSI, MOSEI, and SIMS), providing additional uniformity, transparency, and fairness compared to existing evaluations in the literature. Empirically, LNLN consistently outperforms existing baselines, demonstrating superior performance across these challenging and extensive evaluation metrics.","url_abs":"https://arxiv.org/abs/2409.20012v2","url_pdf":"https://arxiv.org/pdf/2409.20012v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-robust-multimodal-sentiment-analysis","repo_url":"https://github.com/haoyu-ha/lnln","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"multimodal-sentiment-analysis","task_name":"Multimodal Sentiment Analysis"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2409.20012","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.20012"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/haoyu-ha/lnln","reach":null}],"summary":{"ran":9,"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":14,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8ced8002b277acce","entry":"Attention","repo":"haoyu-ha/lnln","repo_kind":"official","path":"models/lnln.py","file_url":"https://github.com/haoyu-ha/lnln/blob/HEAD/models/lnln.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8ced8002b277acce"}},{"code_sha256_prefix":"9479a95a1709394e","entry":"CrossTransformerEncoder","repo":"haoyu-ha/lnln","repo_kind":"official","path":"models/lnln.py","file_url":"https://github.com/haoyu-ha/lnln/blob/HEAD/models/lnln.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9479a95a1709394e"}},{"code_sha256_prefix":"2c0d4b2a4a933000","entry":"GradientReversalLayer","repo":"haoyu-ha/lnln","repo_kind":"official","path":"models/lnln.py","file_url":"https://github.com/haoyu-ha/lnln/blob/HEAD/models/lnln.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2c0d4b2a4a933000"}},{"code_sha256_prefix":"b7ac1eda234ca7a0","entry":"HhyperLearningEncoder","repo":"haoyu-ha/lnln","repo_kind":"official","path":"models/lnln.py","file_url":"https://github.com/haoyu-ha/lnln/blob/HEAD/models/lnln.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b7ac1eda234ca7a0"}},{"code_sha256_prefix":"6a3c9761d14b858c","entry":"HhyperLearningLayer","repo":"haoyu-ha/lnln","repo_kind":"official","path":"models/lnln.py","file_url":"https://github.com/haoyu-ha/lnln/blob/HEAD/models/lnln.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6a3c9761d14b858c"}},{"code_sha256_prefix":"670781bc5235810d","entry":"PreNorm_hyper","repo":"haoyu-ha/lnln","repo_kind":"official","path":"models/lnln.py","file_url":"https://github.com/haoyu-ha/lnln/blob/HEAD/models/lnln.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"670781bc5235810d"}},{"code_sha256_prefix":"82683b7abf1c3e89","entry":"PreNorm_qkv","repo":"haoyu-ha/lnln","repo_kind":"official","path":"models/lnln.py","file_url":"https://github.com/haoyu-ha/lnln/blob/HEAD/models/lnln.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"82683b7abf1c3e89"}},{"code_sha256_prefix":"4e3cc466dd44fcfa","entry":"Transformer","repo":"haoyu-ha/lnln","repo_kind":"official","path":"models/lnln.py","file_url":"https://github.com/haoyu-ha/lnln/blob/HEAD/models/lnln.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4e3cc466dd44fcfa"}},{"code_sha256_prefix":"a5f60555f90d2b05","entry":"TransformerEncoder","repo":"haoyu-ha/lnln","repo_kind":"official","path":"models/lnln.py","file_url":"https://github.com/haoyu-ha/lnln/blob/HEAD/models/lnln.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a5f60555f90d2b05"}},{"code_sha256_prefix":"980a7a91e8c19f50","entry":"evaluate","repo":"Haoyu-ha/LNLN","repo_kind":"official","path":"robust_evaluation.py","file_url":"https://github.com/Haoyu-ha/LNLN/blob/HEAD/robust_evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"980a7a91e8c19f50"}},{"code_sha256_prefix":"70bc37d0c6b440f1","entry":"BertTextEncoder","repo":"haoyu-ha/lnln","repo_kind":"official","path":"models/lnln.py","file_url":"https://github.com/haoyu-ha/lnln/blob/HEAD/models/lnln.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"70bc37d0c6b440f1"}},{"code_sha256_prefix":"a894a9c195e324dd","entry":"CrossTransformer","repo":"haoyu-ha/lnln","repo_kind":"official","path":"models/lnln.py","file_url":"https://github.com/haoyu-ha/lnln/blob/HEAD/models/lnln.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a894a9c195e324dd"}},{"code_sha256_prefix":"c891ae581eab6627","entry":"GradientReversalFn","repo":"haoyu-ha/lnln","repo_kind":"official","path":"models/lnln.py","file_url":"https://github.com/haoyu-ha/lnln/blob/HEAD/models/lnln.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c891ae581eab6627"}},{"code_sha256_prefix":"68bac151f1ad1031","entry":"LNLN","repo":"haoyu-ha/lnln","repo_kind":"official","path":"models/lnln.py","file_url":"https://github.com/haoyu-ha/lnln/blob/HEAD/models/lnln.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"68bac151f1ad1031"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}