{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mygo-discrete-modality-information-as-fine","title":"Tokenization, Fusion, and Augmentation: Towards Fine-grained Multi-modal Entity Representation","arxiv_id":"2404.09468","date":"2024-04-15","proceeding":null,"authors":["Yichi Zhang","Zhuo Chen","Lingbing Guo","Yajing Xu","Binbin Hu","Ziqi Liu","Wen Zhang","Huajun Chen"],"abstract":"Multi-modal knowledge graph completion (MMKGC) aims to discover unobserved knowledge from given knowledge graphs, collaboratively leveraging structural information from the triples and multi-modal information of the entities to overcome the inherent incompleteness. Existing MMKGC methods usually extract multi-modal features with pre-trained models, resulting in coarse handling of multi-modal entity information, overlooking the nuanced, fine-grained semantic details and their complex interactions. To tackle this shortfall, we introduce a novel framework MyGO to tokenize, fuse, and augment the fine-grained multi-modal representations of entities and enhance the MMKGC performance. Motivated by the tokenization technology, MyGO tokenizes multi-modal entity information as fine-grained discrete tokens and learns entity representations with a cross-modal entity encoder. To further augment the multi-modal representations, MyGO incorporates fine-grained contrastive learning to highlight the specificity of the entity representations. Experiments on standard MMKGC benchmarks reveal that our method surpasses 19 of the latest models, underlining its superior performance. Code and data can be found in https://github.com/zjukg/MyGO","url_abs":"https://arxiv.org/abs/2404.09468v2","url_pdf":"https://arxiv.org/pdf/2404.09468v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mygo-discrete-modality-information-as-fine","repo_url":"https://github.com/zjukg/mygo","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"mygo-discrete-modality-information-as-fine","repo_url":"https://github.com/zjukg/adamf-mat","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"descriptive","task_name":"Descriptive"},{"task_slug":"knowledge-graph-completion","task_name":"Knowledge Graph Completion"},{"task_slug":"knowledge-graphs","task_name":"Knowledge Graphs"},{"task_slug":"multi-modal-knowledge-graph","task_name":"Multi-modal Knowledge Graph"},{"task_slug":"specificity","task_name":"Specificity"},{"task_slug":"world-knowledge","task_name":"World Knowledge"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2404.09468","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}