{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-on-large-scale-text-attributed","title":"Learning on Large-scale Text-attributed Graphs via Variational Inference","arxiv_id":"2210.14709","date":"2022-10-26","proceeding":null,"authors":["Jianan Zhao","Meng Qu","Chaozhuo Li","Hao Yan","Qian Liu","Rui Li","Xing Xie","Jian Tang"],"abstract":"This paper studies learning on text-attributed graphs (TAGs), where each node is associated with a text description. An ideal solution for such a problem would be integrating both the text and graph structure information with large language models and graph neural networks (GNNs). However, the problem becomes very challenging when graphs are large due to the high computational complexity brought by training large language models and GNNs together. In this paper, we propose an efficient and effective solution to learning on large text-attributed graphs by fusing graph structure and language learning with a variational Expectation-Maximization (EM) framework, called GLEM. Instead of simultaneously training large language models and GNNs on big graphs, GLEM proposes to alternatively update the two modules in the E-step and M-step. Such a procedure allows training the two modules separately while simultaneously allowing the two modules to interact and mutually enhance each other. Extensive experiments on multiple data sets demonstrate the efficiency and effectiveness of the proposed approach.","url_abs":"https://arxiv.org/abs/2210.14709v2","url_pdf":"https://arxiv.org/pdf/2210.14709v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-on-large-scale-text-attributed","repo_url":"https://github.com/andyjzhao/glem","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"learning-on-large-scale-text-attributed","repo_url":"https://github.com/AndyJZhao/GLEM","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/node-property-prediction-on-ogbn-arxiv","task":"Node Property Prediction","dataset":"ogbn-arxiv","model":"GLEM+RevGAT","rank_in_archive_order":7,"of":86,"metrics":{"Ext. data":"Yes","Number of params":"140469624","Test Accuracy":"0.7694 ± 0.0025","Validation Accuracy":"0.7746 ± 0.0018"},"uses_additional_data":false},{"leaderboard":"/sota/node-property-prediction-on-ogbn-papers100m","task":"Node Property Prediction","dataset":"ogbn-papers100M","model":"GLEM+GIANT+GAMLP","rank_in_archive_order":1,"of":20,"metrics":{"Ext. data":"Yes","Number of params":"154775375","Test Accuracy":"0.7037 ± 0.0002","Validation Accuracy":"0.7354 ± 0.0001"},"uses_additional_data":false},{"leaderboard":"/sota/node-property-prediction-on-ogbn-products","task":"Node Property Prediction","dataset":"ogbn-products","model":"GLEM+EnGCN","rank_in_archive_order":1,"of":64,"metrics":{"Ext. data":"Yes","Number of params":"139633805","Test Accuracy":"0.9014 ± 0.0012","Validation Accuracy":"0.9370 ± 0.0004"},"uses_additional_data":false},{"leaderboard":"/sota/node-property-prediction-on-ogbn-products","task":"Node Property Prediction","dataset":"ogbn-products","model":"GLEM+GIANT+SAGN+SCR","rank_in_archive_order":3,"of":64,"metrics":{"Ext. data":"Yes","Number of params":"139792525","Test Accuracy":"0.8737 ± 0.0006","Validation Accuracy":"0.9400 ± 0.0003"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2210.14709","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.14709"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/andyjzhao/glem","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/AndyJZhao/GLEM","reach":{"status":"ok"}}],"summary":{"unverified":1},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"005cfd689309ecee","entry":"GLEMTrainer","repo":"AndyJZhao/GLEM","repo_kind":"official","path":"src/models/GLEM/GLEM_trainer.py","file_url":"https://github.com/AndyJZhao/GLEM/blob/HEAD/src/models/GLEM/GLEM_trainer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"005cfd689309ecee"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}