{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unigraph-learning-a-cross-domain-graph","title":"UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs","arxiv_id":"2402.13630","date":"2024-02-21","proceeding":null,"authors":["Yufei He","Yuan Sui","Xiaoxin He","Bryan Hooi"],"abstract":"Foundation models like ChatGPT and GPT-4 have revolutionized artificial intelligence, exhibiting remarkable abilities to generalize across a wide array of tasks and applications beyond their initial training objectives. However, graph learning has predominantly focused on single-graph models, tailored to specific tasks or datasets, lacking the ability to transfer learned knowledge to different domains. This limitation stems from the inherent complexity and diversity of graph structures, along with the different feature and label spaces specific to graph data. In this paper, we recognize text as an effective unifying medium and employ Text-Attributed Graphs (TAGs) to leverage this potential. We present our UniGraph framework, designed to learn a foundation model for TAGs, which is capable of generalizing to unseen graphs and tasks across diverse domains. Unlike single-graph models that use pre-computed node features of varying dimensions as input, our approach leverages textual features for unifying node representations, even for graphs such as molecular graphs that do not naturally have textual features. We propose a novel cascaded architecture of Language Models (LMs) and Graph Neural Networks (GNNs) as backbone networks. Additionally, we propose the first pre-training algorithm specifically designed for large-scale self-supervised learning on TAGs, based on Masked Graph Modeling. We introduce graph instruction tuning using Large Language Models (LLMs) to enable zero-shot prediction ability. Our comprehensive experiments across various graph learning tasks and domains demonstrate the model's effectiveness in self-supervised representation learning on unseen graphs, few-shot in-context transfer, and zero-shot transfer, even surpassing or matching the performance of GNNs that have undergone supervised training on target datasets.","url_abs":"https://arxiv.org/abs/2402.13630v3","url_pdf":"https://arxiv.org/pdf/2402.13630v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unigraph-learning-a-cross-domain-graph","repo_url":"https://github.com/yf-he/UniGraph","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"graph-learning","task_name":"Graph Learning"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-4","method_name":"GPT-4"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.13630","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.13630"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yf-he/UniGraph","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1,"ran":4,"unverified":5},"by_repo_kind":{"official":{"samples":10,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"adf3b9c6c22af5ce","entry":"accuracy","repo":"yf-he/UniGraph","repo_kind":"official","path":"utils/evaluation.py","file_url":"https://github.com/yf-he/UniGraph/blob/HEAD/utils/evaluation.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"adf3b9c6c22af5ce"}},{"code_sha256_prefix":"b36b7319ea433429","entry":"create_optimizer","repo":"yf-he/UniGraph","repo_kind":"official","path":"utils/evaluation.py","file_url":"https://github.com/yf-he/UniGraph/blob/HEAD/utils/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b36b7319ea433429"}},{"code_sha256_prefix":"63d4d8375f9f4fca","entry":"mask_edge","repo":"yf-he/UniGraph","repo_kind":"official","path":"utils/functions.py","file_url":"https://github.com/yf-he/UniGraph/blob/HEAD/utils/functions.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"63d4d8375f9f4fca"}},{"code_sha256_prefix":"79e8f00e0955cc92","entry":"pth_safe_load","repo":"yf-he/UniGraph","repo_kind":"official","path":"data.py","file_url":"https://github.com/yf-he/UniGraph/blob/HEAD/data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"79e8f00e0955cc92"}},{"code_sha256_prefix":"11cf69e6642b61f5","entry":"scale_feats","repo":"yf-he/UniGraph","repo_kind":"official","path":"utils/data_util.py","file_url":"https://github.com/yf-he/UniGraph/blob/HEAD/utils/data_util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"11cf69e6642b61f5"}},{"code_sha256_prefix":"c9c9174dc014b586","entry":"get_current_lr","repo":"yf-he/UniGraph","repo_kind":"official","path":"utils/functions.py","file_url":"https://github.com/yf-he/UniGraph/blob/HEAD/utils/functions.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c9c9174dc014b586"}},{"code_sha256_prefix":"e7ebdfbc2b843a1b","entry":"get_ego_subgraphs","repo":"yf-he/UniGraph","repo_kind":"official","path":"data.py","file_url":"https://github.com/yf-he/UniGraph/blob/HEAD/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e7ebdfbc2b843a1b"}},{"code_sha256_prefix":"cb6e44a7cc7082be","entry":"node_classification_evaluation","repo":"yf-he/UniGraph","repo_kind":"official","path":"utils/evaluation.py","file_url":"https://github.com/yf-he/UniGraph/blob/HEAD/utils/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cb6e44a7cc7082be"}},{"code_sha256_prefix":"6f9bb5eb98ec4887","entry":"sce_loss","repo":"yf-he/UniGraph","repo_kind":"official","path":"utils/loss_func.py","file_url":"https://github.com/yf-he/UniGraph/blob/HEAD/utils/loss_func.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6f9bb5eb98ec4887"}},{"code_sha256_prefix":"f2502a1d225b3dd3","entry":"sig_loss","repo":"yf-he/UniGraph","repo_kind":"official","path":"utils/loss_func.py","file_url":"https://github.com/yf-he/UniGraph/blob/HEAD/utils/loss_func.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f2502a1d225b3dd3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}