{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/uniglm-training-one-unified-language-model","title":"UniGLM: Training One Unified Language Model for Text-Attributed Graph Embedding","arxiv_id":"2406.12052","date":"2024-06-17","proceeding":null,"authors":["Yi Fang","Dongzhe Fan","Sirui Ding","Ninghao Liu","Qiaoyu Tan"],"abstract":"Representation learning on text-attributed graphs (TAGs), where nodes are represented by textual descriptions, is crucial for textual and relational knowledge systems and recommendation systems. Currently, state-of-the-art embedding methods for TAGs primarily focus on fine-tuning language models (e.g., BERT) using structure-aware training signals. While effective, these methods are tailored for individual TAG and cannot generalize across various graph scenarios. Given the shared textual space, leveraging multiple TAGs for joint fine-tuning, aligning text and graph structure from different aspects, would be more beneficial. Motivated by this, we introduce a novel Unified Graph Language Model (UniGLM) framework, the first graph embedding model that generalizes well to both in-domain and cross-domain TAGs. Specifically, UniGLM is trained over multiple TAGs with different domains and scales using self-supervised contrastive learning. UniGLM includes an adaptive positive sample selection technique for identifying structurally similar nodes and a lazy contrastive module that is devised to accelerate training by minimizing repetitive encoding calculations. Extensive empirical results across 9 benchmark TAGs demonstrate UniGLM's efficacy against leading embedding baselines in terms of generalization (various downstream tasks and backbones) and transfer learning (in and out of domain scenarios). The code is available at https://github.com/NYUSHCS/UniGLM.","url_abs":"https://arxiv.org/abs/2406.12052v2","url_pdf":"https://arxiv.org/pdf/2406.12052v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"uniglm-training-one-unified-language-model","repo_url":"https://github.com/nyushcs/uniglm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"graph-embedding","task_name":"Graph Embedding"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"recommendation-systems","task_name":"Recommendation Systems"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"tag","task_name":"TAG"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2406.12052","atlas_url":"https://app.syntology.ai/?focus=2406.12052","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.12052"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nyushcs/uniglm","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"587bcdd318e3716e","entry":"get_dir_of_file","repo":"nyushcs/uniglm","repo_kind":"official","path":"core/GNNs/utils.py","file_url":"https://github.com/nyushcs/uniglm/blob/HEAD/core/GNNs/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"587bcdd318e3716e"}},{"code_sha256_prefix":"a0a4a124393e3929","entry":"init_path","repo":"nyushcs/uniglm","repo_kind":"official","path":"core/GNNs/utils.py","file_url":"https://github.com/nyushcs/uniglm/blob/HEAD/core/GNNs/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a0a4a124393e3929"}},{"code_sha256_prefix":"41568b1c9d167a57","entry":"prep_edges","repo":"nyushcs/uniglm","repo_kind":"official","path":"core/GNNs/TrainerLP.py","file_url":"https://github.com/nyushcs/uniglm/blob/HEAD/core/GNNs/TrainerLP.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"41568b1c9d167a57"}},{"code_sha256_prefix":"e665796b8abfaf3c","entry":"split_edges","repo":"nyushcs/uniglm","repo_kind":"official","path":"core/GNNs/TrainerLP.py","file_url":"https://github.com/nyushcs/uniglm/blob/HEAD/core/GNNs/TrainerLP.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e665796b8abfaf3c"}},{"code_sha256_prefix":"52f69633f57cef09","entry":"time2str","repo":"nyushcs/uniglm","repo_kind":"official","path":"core/GNNs/utils.py","file_url":"https://github.com/nyushcs/uniglm/blob/HEAD/core/GNNs/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"52f69633f57cef09"}},{"code_sha256_prefix":"7969292711261485","entry":"get_gnn_trainer","repo":"nyushcs/uniglm","repo_kind":"official","path":"core/GNNs/GNN_utils.py","file_url":"https://github.com/nyushcs/uniglm/blob/HEAD/core/GNNs/GNN_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7969292711261485"}},{"code_sha256_prefix":"4768a3880037384c","entry":"update_cfg","repo":"nyushcs/uniglm","repo_kind":"official","path":"core/GNNs/config.py","file_url":"https://github.com/nyushcs/uniglm/blob/HEAD/core/GNNs/config.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4768a3880037384c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}