{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/simmmdg-a-simple-and-effective-framework-for-1","title":"SimMMDG: A Simple and Effective Framework for Multi-modal Domain Generalization","arxiv_id":"2310.19795","date":"2023-10-30","proceeding":"NeurIPS 2023 11","authors":["Hao Dong","Ismail Nejjar","Han Sun","Eleni Chatzi","Olga Fink"],"abstract":"In real-world scenarios, achieving domain generalization (DG) presents significant challenges as models are required to generalize to unknown target distributions. Generalizing to unseen multi-modal distributions poses even greater difficulties due to the distinct properties exhibited by different modalities. To overcome the challenges of achieving domain generalization in multi-modal scenarios, we propose SimMMDG, a simple yet effective multi-modal DG framework. We argue that mapping features from different modalities into the same embedding space impedes model generalization. To address this, we propose splitting the features within each modality into modality-specific and modality-shared components. We employ supervised contrastive learning on the modality-shared features to ensure they possess joint properties and impose distance constraints on modality-specific features to promote diversity. In addition, we introduce a cross-modal translation module to regularize the learned features, which can also be used for missing-modality generalization. We demonstrate that our framework is theoretically well-supported and achieves strong performance in multi-modal DG on the EPIC-Kitchens dataset and the novel Human-Animal-Cartoon (HAC) dataset introduced in this paper. Our source code and HAC dataset are available at https://github.com/donghao51/SimMMDG.","url_abs":"https://arxiv.org/abs/2310.19795v1","url_pdf":"https://arxiv.org/pdf/2310.19795v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"simmmdg-a-simple-and-effective-framework-for-1","repo_url":"https://github.com/donghao51/simmmdg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"domain-generalization","task_name":"Domain Generalization"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[{"slug":"human-animal-cartoon","name":"Human-Animal-Cartoon","full_name":"Human-Animal-Cartoon"}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.19795","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.19795"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/donghao51/SimMMDG","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/donghao51/simmmdg","reach":{"status":"ok"}}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"8e0e0a55c8f2f231","entry":"train_one_step","repo":"donghao51/SimMMDG","repo_kind":"official","path":"EPIC-rgb-flow-audio/train_video_flow_audio_EPIC_SimMMDG.py","file_url":"https://github.com/donghao51/SimMMDG/blob/HEAD/EPIC-rgb-flow-audio/train_video_flow_audio_EPIC_SimMMDG.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8e0e0a55c8f2f231"}},{"code_sha256_prefix":"98ce88aa0f481963","entry":"validate_one_step","repo":"donghao51/SimMMDG","repo_kind":"official","path":"EPIC-rgb-flow-audio/train_video_flow_audio_EPIC_SimMMDG.py","file_url":"https://github.com/donghao51/SimMMDG/blob/HEAD/EPIC-rgb-flow-audio/train_video_flow_audio_EPIC_SimMMDG.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"98ce88aa0f481963"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}