{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cooperative-minibatching-in-graph-neural","title":"Cooperative Minibatching in Graph Neural Networks","arxiv_id":"2310.12403","date":"2023-10-19","proceeding":null,"authors":["Muhammed Fatih Balin","Dominique LaSalle","Ümit V. Çatalyürek"],"abstract":"Training large scale Graph Neural Networks (GNNs) requires significant computational resources, and the process is highly data-intensive. One of the most effective ways to reduce resource requirements is minibatch training coupled with graph sampling. GNNs have the unique property that items in a minibatch have overlapping data. However, the commonly implemented Independent Minibatching approach assigns each Processing Element (PE, i.e., cores and/or GPUs) its own minibatch to process, leading to duplicated computations and input data access across PEs. This amplifies the Neighborhood Explosion Phenomenon (NEP), which is the main bottleneck limiting scaling. To reduce the effects of NEP in the multi-PE setting, we propose a new approach called Cooperative Minibatching. Our approach capitalizes on the fact that the size of the sampled subgraph is a concave function of the batch size, leading to significant reductions in the amount of work as batch sizes increase. Hence, it is favorable for processors equipped with a fast interconnect to work on a large minibatch together as a single larger processor, instead of working on separate smaller minibatches, even though global batch size is identical. We also show how to take advantage of the same phenomenon in serial execution by generating dependent consecutive minibatches. Our experimental evaluations show up to 4x bandwidth savings for fetching vertex embeddings, by simply increasing this dependency without harming model convergence. Combining our proposed approaches, we achieve up to 64% speedup over Independent Minibatching on single-node multi-GPU systems, using same resources.","url_abs":"https://arxiv.org/abs/2310.12403v3","url_pdf":"https://arxiv.org/pdf/2310.12403v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cooperative-minibatching-in-graph-neural","repo_url":"https://github.com/gt-tdalab/dgl-coop","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":null,"task_name":"GPU"},{"task_slug":"graph-sampling","task_name":"Graph Sampling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2310.12403","atlas_url":"https://app.syntology.ai/?focus=2310.12403","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.12403"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gt-tdalab/dgl-coop","reach":null}],"summary":{"ran":1,"unverified":8},"by_repo_kind":{"official":{"samples":9,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6ff1ab6414e7e081","entry":"LazyFeature","repo":"gt-tdalab/dgl-coop","repo_kind":"official","path":"python/dgl/dataloading/labor_sampler.py","file_url":"https://github.com/gt-tdalab/dgl-coop/blob/HEAD/python/dgl/dataloading/labor_sampler.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6ff1ab6414e7e081"}},{"code_sha256_prefix":"de458b1316b1fe0a","entry":"BlockSampler","repo":"gt-tdalab/dgl-coop","repo_kind":"official","path":"python/dgl/dataloading/labor_sampler.py","file_url":"https://github.com/gt-tdalab/dgl-coop/blob/HEAD/python/dgl/dataloading/labor_sampler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"de458b1316b1fe0a"}},{"code_sha256_prefix":"9c87070ebf6e4293","entry":"LaborSampler","repo":"gt-tdalab/dgl-coop","repo_kind":"official","path":"python/dgl/dataloading/labor_sampler.py","file_url":"https://github.com/gt-tdalab/dgl-coop/blob/HEAD/python/dgl/dataloading/labor_sampler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9c87070ebf6e4293"}},{"code_sha256_prefix":"228613fed63ef133","entry":"Sampler","repo":"gt-tdalab/dgl-coop","repo_kind":"official","path":"python/dgl/dataloading/labor_sampler.py","file_url":"https://github.com/gt-tdalab/dgl-coop/blob/HEAD/python/dgl/dataloading/labor_sampler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"228613fed63ef133"}},{"code_sha256_prefix":"ff1a37b98c3aa234","entry":"_set_lazy_features","repo":"gt-tdalab/dgl-coop","repo_kind":"official","path":"python/dgl/dataloading/labor_sampler.py","file_url":"https://github.com/gt-tdalab/dgl-coop/blob/HEAD/python/dgl/dataloading/labor_sampler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ff1a37b98c3aa234"}},{"code_sha256_prefix":"7dc7bd1ca50e4bf6","entry":"choice","repo":"gt-tdalab/dgl-coop","repo_kind":"official","path":"python/dgl/dataloading/labor_sampler.py","file_url":"https://github.com/gt-tdalab/dgl-coop/blob/HEAD/python/dgl/dataloading/labor_sampler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7dc7bd1ca50e4bf6"}},{"code_sha256_prefix":"b1bea8639ec3c94d","entry":"set_dst_lazy_features","repo":"gt-tdalab/dgl-coop","repo_kind":"official","path":"python/dgl/dataloading/labor_sampler.py","file_url":"https://github.com/gt-tdalab/dgl-coop/blob/HEAD/python/dgl/dataloading/labor_sampler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b1bea8639ec3c94d"}},{"code_sha256_prefix":"807604b26b54e099","entry":"set_edge_lazy_features","repo":"gt-tdalab/dgl-coop","repo_kind":"official","path":"python/dgl/dataloading/labor_sampler.py","file_url":"https://github.com/gt-tdalab/dgl-coop/blob/HEAD/python/dgl/dataloading/labor_sampler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"807604b26b54e099"}},{"code_sha256_prefix":"a6432514a13e95ab","entry":"set_src_lazy_features","repo":"gt-tdalab/dgl-coop","repo_kind":"official","path":"python/dgl/dataloading/labor_sampler.py","file_url":"https://github.com/gt-tdalab/dgl-coop/blob/HEAD/python/dgl/dataloading/labor_sampler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a6432514a13e95ab"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}