{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/simteg-a-frustratingly-simple-approach","title":"SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning","arxiv_id":"2308.02565","date":"2023-08-03","proceeding":null,"authors":["Keyu Duan","Qian Liu","Tat-Seng Chua","Shuicheng Yan","Wei Tsang Ooi","Qizhe Xie","Junxian He"],"abstract":"Textual graphs (TGs) are graphs whose nodes correspond to text (sentences or documents), which are widely prevalent. The representation learning of TGs involves two stages: (i) unsupervised feature extraction and (ii) supervised graph representation learning. In recent years, extensive efforts have been devoted to the latter stage, where Graph Neural Networks (GNNs) have dominated. However, the former stage for most existing graph benchmarks still relies on traditional feature engineering techniques. More recently, with the rapid development of language models (LMs), researchers have focused on leveraging LMs to facilitate the learning of TGs, either by jointly training them in a computationally intensive framework (merging the two stages), or designing complex self-supervised training tasks for feature extraction (enhancing the first stage). In this work, we present SimTeG, a frustratingly Simple approach for Textual Graph learning that does not innovate in frameworks, models, and tasks. Instead, we first perform supervised parameter-efficient fine-tuning (PEFT) on a pre-trained LM on the downstream task, such as node classification. We then generate node embeddings using the last hidden states of finetuned LM. These derived features can be further utilized by any GNN for training on the same task. We evaluate our approach on two fundamental graph representation learning tasks: node classification and link prediction. Through extensive experiments, we show that our approach significantly improves the performance of various GNNs on multiple graph benchmarks.","url_abs":"https://arxiv.org/abs/2308.02565v1","url_pdf":"https://arxiv.org/pdf/2308.02565v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"simteg-a-frustratingly-simple-approach","repo_url":"https://github.com/vermouthdky/simteg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"simteg-a-frustratingly-simple-approach","repo_url":"https://github.com/vermouthdky/SimTeG","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"feature-engineering","task_name":"Feature Engineering"},{"task_slug":"graph-learning","task_name":"Graph Learning"},{"task_slug":"graph-representation-learning","task_name":"Graph Representation Learning"},{"task_slug":"link-prediction","task_name":"Link Prediction"},{"task_slug":"node-classification","task_name":"Node Classification"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"parameter-efficient-fine-tuning","task_name":"parameter-efficient fine-tuning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/node-property-prediction-on-ogbn-arxiv","task":"Node Property Prediction","dataset":"ogbn-arxiv","model":"SimTeG+TAPE+RevGAT","rank_in_archive_order":2,"of":86,"metrics":{"Ext. data":"Yes","Number of params":"1386219488","Test Accuracy":"0.7803 ± 0.0007","Validation Accuracy":"0.7846 ± 0.0004"},"uses_additional_data":false},{"leaderboard":"/sota/node-property-prediction-on-ogbn-arxiv","task":"Node Property Prediction","dataset":"ogbn-arxiv","model":"SimTeG+TAPE+GraphSAGE","rank_in_archive_order":4,"of":86,"metrics":{"Ext. data":"Yes","Number of params":"1381593403","Test Accuracy":"0.7748 ± 0.0011","Validation Accuracy":"0.7789 ± 0.0008"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2308.02565","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.02565"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vermouthdky/simteg","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vermouthdky/SimTeG","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"47e3e3644016d7b9","entry":"dataset2foldername","repo":"vermouthdky/simteg","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/vermouthdky/simteg/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"47e3e3644016d7b9"}},{"code_sha256_prefix":"23459a00ee3118da","entry":"dict_append","repo":"vermouthdky/simteg","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/vermouthdky/simteg/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"23459a00ee3118da"}},{"code_sha256_prefix":"9cc3593e6d666fb6","entry":"load_args","repo":"vermouthdky/simteg","repo_kind":"official","path":"src/args.py","file_url":"https://github.com/vermouthdky/simteg/blob/HEAD/src/args.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9cc3593e6d666fb6"}},{"code_sha256_prefix":"119e4fa27670079b","entry":"correct_and_smooth","repo":"vermouthdky/simteg","repo_kind":"official","path":"compute_ensemble.py","file_url":"https://github.com/vermouthdky/simteg/blob/HEAD/compute_ensemble.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"119e4fa27670079b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}