{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-large-graph-property-prediction-via","title":"Learning Large Graph Property Prediction via Graph Segment Training","arxiv_id":"2305.12322","date":"2023-05-21","proceeding":"NeurIPS 2023 11","authors":["Kaidi Cao","Phitchaya Mangpo Phothilimthana","Sami Abu-El-Haija","Dustin Zelle","Yanqi Zhou","Charith Mendis","Jure Leskovec","Bryan Perozzi"],"abstract":"Learning to predict properties of large graphs is challenging because each prediction requires the knowledge of an entire graph, while the amount of memory available during training is bounded. Here we propose Graph Segment Training (GST), a general framework that utilizes a divide-and-conquer approach to allow learning large graph property prediction with a constant memory footprint. GST first divides a large graph into segments and then backpropagates through only a few segments sampled per training iteration. We refine the GST paradigm by introducing a historical embedding table to efficiently obtain embeddings for segments not sampled for backpropagation. To mitigate the staleness of historical embeddings, we design two novel techniques. First, we finetune the prediction head to fix the input distribution shift. Second, we introduce Stale Embedding Dropout to drop some stale embeddings during training to reduce bias. We evaluate our complete method GST-EFD (with all the techniques together) on two large graph property prediction benchmarks: MalNet and TpuGraphs. Our experiments show that GST-EFD is both memory-efficient and fast, while offering a slight boost on test accuracy over a typical full graph training regime.","url_abs":"https://arxiv.org/abs/2305.12322v3","url_pdf":"https://arxiv.org/pdf/2305.12322v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-large-graph-property-prediction-via","repo_url":"https://github.com/kaidic/gst","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"graph-property-prediction","task_name":"Graph Property Prediction"},{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"property-prediction","task_name":"Property Prediction"}],"methods":[{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"embedding-dropout","method_name":"Embedding Dropout"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.12322","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.12322"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kaidic/gst","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_violates":2,"ran":4,"unverified":2},"by_repo_kind":{"official":{"samples":8,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"60fff7c3c400d7ff","entry":"default","repo":"kaidic/gst","repo_kind":"official","path":"graphgps/layer/performer_layer.py","file_url":"https://github.com/kaidic/gst/blob/HEAD/graphgps/layer/performer_layer.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"60fff7c3c400d7ff"}},{"code_sha256_prefix":"19925529d217754e","entry":"empty","repo":"kaidic/gst","repo_kind":"official","path":"graphgps/layer/performer_layer.py","file_url":"https://github.com/kaidic/gst/blob/HEAD/graphgps/layer/performer_layer.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"19925529d217754e"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"kaidic/gst","repo_kind":"official","path":"graphgps/layer/performer_layer.py","file_url":"https://github.com/kaidic/gst/blob/HEAD/graphgps/layer/performer_layer.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"1bb331bf70a78136","entry":"get_final_pretrained_ckpt","repo":"kaidic/gst","repo_kind":"official","path":"graphgps/finetuning.py","file_url":"https://github.com/kaidic/gst/blob/HEAD/graphgps/finetuning.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1bb331bf70a78136"}},{"code_sha256_prefix":"9e4db46ba20af98e","entry":"init_model_from_pretrained","repo":"kaidic/gst","repo_kind":"official","path":"graphgps/finetuning.py","file_url":"https://github.com/kaidic/gst/blob/HEAD/graphgps/finetuning.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9e4db46ba20af98e"}},{"code_sha256_prefix":"585566394225e457","entry":"new_optimizer_config","repo":"kaidic/gst","repo_kind":"official","path":"main_tpugraphs.py","file_url":"https://github.com/kaidic/gst/blob/HEAD/main_tpugraphs.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"585566394225e457"}},{"code_sha256_prefix":"ae56bcfaae96da3c","entry":"apply_chunking_to_forward","repo":"kaidic/gst","repo_kind":"official","path":"graphgps/layer/bigbird_layer.py","file_url":"https://github.com/kaidic/gst/blob/HEAD/graphgps/layer/bigbird_layer.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ae56bcfaae96da3c"}},{"code_sha256_prefix":"a969312556ec39bb","entry":"load_pretrained_model_cfg","repo":"kaidic/gst","repo_kind":"official","path":"graphgps/finetuning.py","file_url":"https://github.com/kaidic/gst/blob/HEAD/graphgps/finetuning.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a969312556ec39bb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}