{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/modality-independent-graph-neural-networks","title":"Modality-Independent Graph Neural Networks with Global Transformers for Multimodal Recommendation","arxiv_id":"2412.13994","date":"2024-12-18","proceeding":null,"authors":["Jun Hu","Bryan Hooi","Bingsheng He","Yinwei Wei"],"abstract":"Multimodal recommendation systems can learn users' preferences from existing user-item interactions as well as the semantics of multimodal data associated with items. Many existing methods model this through a multimodal user-item graph, approaching multimodal recommendation as a graph learning task. Graph Neural Networks (GNNs) have shown promising performance in this domain. Prior research has capitalized on GNNs' capability to capture neighborhood information within certain receptive fields (typically denoted by the number of hops, $K$) to enrich user and item semantics. We observe that the optimal receptive fields for GNNs can vary across different modalities. In this paper, we propose GNNs with Modality-Independent Receptive Fields, which employ separate GNNs with independent receptive fields for different modalities to enhance performance. Our results indicate that the optimal $K$ for certain modalities on specific datasets can be as low as 1 or 2, which may restrict the GNNs' capacity to capture global information. To address this, we introduce a Sampling-based Global Transformer, which utilizes uniform global sampling to effectively integrate global information for GNNs. We conduct comprehensive experiments that demonstrate the superiority of our approach over existing methods. Our code is publicly available at https://github.com/CrawlScript/MIG-GT.","url_abs":"https://arxiv.org/abs/2412.13994v1","url_pdf":"https://arxiv.org/pdf/2412.13994v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"modality-independent-graph-neural-networks","repo_url":"https://github.com/crawlscript/mig-gt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"graph-learning","task_name":"Graph Learning"},{"task_slug":"multi-modal-recommendation","task_name":"Multi-modal Recommendation"},{"task_slug":"multimodal-recommendation","task_name":"Multimodal Recommendation"},{"task_slug":"recommendation-systems","task_name":"Recommendation Systems"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/multi-modal-recommendation-on-amazon-baby","task":"Multi-modal Recommendation","dataset":"Amazon Baby","model":"MIG-GT","rank_in_archive_order":1,"of":10,"metrics":{"NDCG@20":"0.0452"},"uses_additional_data":false},{"leaderboard":"/sota/multi-modal-recommendation-on-amazon-clothing","task":"Multi-modal Recommendation","dataset":"Amazon Clothing","model":"MIG-GT","rank_in_archive_order":1,"of":10,"metrics":{"NDCG@20":"0.0422"},"uses_additional_data":false},{"leaderboard":"/sota/multi-modal-recommendation-on-amazon-sports","task":"Multi-modal Recommendation","dataset":"Amazon Sports","model":"MIG-GT","rank_in_archive_order":1,"of":10,"metrics":{"NGCG@20":"0.0511"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}