{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/imf-interactive-multimodal-fusion-model-for","title":"IMF: Interactive Multimodal Fusion Model for Link Prediction","arxiv_id":"2303.10816","date":"2023-03-20","proceeding":null,"authors":["Xinhang Li","Xiangyu Zhao","Jiaxing Xu","Yong Zhang","Chunxiao Xing"],"abstract":"Link prediction aims to identify potential missing triples in knowledge graphs. To get better results, some recent studies have introduced multimodal information to link prediction. However, these methods utilize multimodal information separately and neglect the complicated interaction between different modalities. In this paper, we aim at better modeling the inter-modality information and thus introduce a novel Interactive Multimodal Fusion (IMF) model to integrate knowledge from different modalities. To this end, we propose a two-stage multimodal fusion framework to preserve modality-specific knowledge as well as take advantage of the complementarity between different modalities. Instead of directly projecting different modalities into a unified space, our multimodal fusion module limits the representations of different modalities independent while leverages bilinear pooling for fusion and incorporates contrastive learning as additional constraints. Furthermore, the decision fusion module delivers the learned weighted average over the predictions of all modalities to better incorporate the complementarity of different modalities. Our approach has been demonstrated to be effective through empirical evaluations on several real-world datasets. The implementation code is available online at https://github.com/HestiaSky/IMF-Pytorch.","url_abs":"https://arxiv.org/abs/2303.10816v1","url_pdf":"https://arxiv.org/pdf/2303.10816v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"imf-interactive-multimodal-fusion-model-for","repo_url":"https://github.com/hestiasky/imf-pytorch","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"knowledge-graphs","task_name":"Knowledge Graphs"},{"task_slug":"link-prediction","task_name":"Link Prediction"},{"task_slug":"prediction","task_name":"Prediction"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2303.10816","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.10816"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hestiasky/imf-pytorch","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fd2799667992935b","entry":"get_adj","repo":"hestiasky/imf-pytorch","repo_kind":"official","path":"utils/data_util.py","file_url":"https://github.com/hestiasky/imf-pytorch/blob/HEAD/utils/data_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fd2799667992935b"}},{"code_sha256_prefix":"3aa2be2a8b7b3fc0","entry":"read_entity_from_id","repo":"hestiasky/imf-pytorch","repo_kind":"official","path":"utils/data_util.py","file_url":"https://github.com/hestiasky/imf-pytorch/blob/HEAD/utils/data_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3aa2be2a8b7b3fc0"}},{"code_sha256_prefix":"f8b47e60f1425118","entry":"read_relation_from_id","repo":"hestiasky/imf-pytorch","repo_kind":"official","path":"utils/data_util.py","file_url":"https://github.com/hestiasky/imf-pytorch/blob/HEAD/utils/data_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f8b47e60f1425118"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}