{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-neural-scaling-laws-for-foundation","title":"Towards Neural Scaling Laws for Foundation Models on Temporal Graphs","arxiv_id":"2406.10426","date":"2024-06-14","proceeding":null,"authors":["Razieh Shirzadkhani","Tran Gia Bao Ngo","Kiarash Shamsi","Shenyang Huang","Farimah Poursafaei","Poupak Azad","Reihaneh Rabbany","Baris Coskunuzer","Guillaume Rabusseau","Cuneyt Gurcan Akcora"],"abstract":"The field of temporal graph learning aims to learn from evolving network data to forecast future interactions. Given a collection of observed temporal graphs, is it possible to predict the evolution of an unseen network from the same domain? To answer this question, we first present the Temporal Graph Scaling (TGS) dataset, a large collection of temporal graphs consisting of eighty-four ERC20 token transaction networks collected from 2017 to 2023. Next, we evaluate the transferability of Temporal Graph Neural Networks (TGNNs) for the temporal graph property prediction task by pre-training on a collection of up to sixty-four token transaction networks and then evaluating the downstream performance on twenty unseen token networks. We find that the neural scaling law observed in NLP and Computer Vision also applies in temporal graph learning, where pre-training on greater number of networks leads to improved downstream performance. To the best of our knowledge, this is the first empirical demonstration of the transferability of temporal graphs learning. On downstream token networks, the largest pre-trained model outperforms single model TGNNs on thirteen unseen test networks. Therefore, we believe that this is a promising first step towards building foundation models for temporal graphs.","url_abs":"https://arxiv.org/abs/2406.10426v2","url_pdf":"https://arxiv.org/pdf/2406.10426v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-neural-scaling-laws-for-foundation","repo_url":"https://github.com/benjaminnNgo/ScalingTGNs","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"graph-learning","task_name":"Graph Learning"},{"task_slug":"graph-property-prediction","task_name":"Graph Property Prediction"},{"task_slug":"property-prediction","task_name":"Property Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2406.10426","atlas_url":"https://app.syntology.ai/?focus=2406.10426","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.10426"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/benjaminnNgo/ScalingTGNs","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0b58c7a4bc69d4dd","entry":"get_node_id_int","repo":"benjaminnNgo/ScalingTGNs","repo_kind":"official","path":"script/gclstm_foundation_model.py","file_url":"https://github.com/benjaminnNgo/ScalingTGNs/blob/HEAD/script/gclstm_foundation_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0b58c7a4bc69d4dd"}},{"code_sha256_prefix":"520dfca725ed1118","entry":"mkdirs","repo":"benjaminnNgo/ScalingTGNs","repo_kind":"official","path":"script/gclstm_foundation_model.py","file_url":"https://github.com/benjaminnNgo/ScalingTGNs/blob/HEAD/script/gclstm_foundation_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"520dfca725ed1118"}},{"code_sha256_prefix":"461e88a09d13753d","entry":"readout_function","repo":"benjaminnNgo/ScalingTGNs","repo_kind":"official","path":"script/gclstm_foundation_model.py","file_url":"https://github.com/benjaminnNgo/ScalingTGNs/blob/HEAD/script/gclstm_foundation_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"461e88a09d13753d"}},{"code_sha256_prefix":"5bb0325907c78f41","entry":"readout_function","repo":"benjaminnNgo/ScalingTGNs","repo_kind":"official","path":"script/train_foundation_tgc_mix.py","file_url":"https://github.com/benjaminnNgo/ScalingTGNs/blob/HEAD/script/train_foundation_tgc_mix.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5bb0325907c78f41"}},{"code_sha256_prefix":"70e9431a2a9d96bd","entry":"extra_dataset_attributes_loading","repo":"benjaminnNgo/ScalingTGNs","repo_kind":"official","path":"script/train_single_tgc.py","file_url":"https://github.com/benjaminnNgo/ScalingTGNs/blob/HEAD/script/train_single_tgc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"70e9431a2a9d96bd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}