{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/topology-aware-embedding-memory-for-learning","title":"Topology-aware Embedding Memory for Continual Learning on Expanding Networks","arxiv_id":"2401.13200","date":"2024-01-24","proceeding":null,"authors":["Xikun Zhang","Dongjin Song","Yixin Chen","DaCheng Tao"],"abstract":"Memory replay based techniques have shown great success for continual learning with incrementally accumulated Euclidean data. Directly applying them to continually expanding networks, however, leads to the potential memory explosion problem due to the need to buffer representative nodes and their associated topological neighborhood structures. To this end, we systematically analyze the key challenges in the memory explosion problem, and present a general framework, \\textit{i.e.}, Parameter Decoupled Graph Neural Networks (PDGNNs) with Topology-aware Embedding Memory (TEM), to tackle this issue. The proposed framework not only reduces the memory space complexity from $\\mathcal{O}(nd^L)$ to $\\mathcal{O}(n)$~\\footnote{$n$: memory budget, $d$: average node degree, $L$: the radius of the GNN receptive field}, but also fully utilizes the topological information for memory replay. Specifically, PDGNNs decouple trainable parameters from the computation ego-subnetwork via \\textit{Topology-aware Embeddings} (TEs), which compress ego-subnetworks into compact vectors (\\textit{i.e.}, TEs) to reduce the memory consumption. Based on this framework, we discover a unique \\textit{pseudo-training effect} in continual learning on expanding networks and this effect motivates us to develop a novel \\textit{coverage maximization sampling} strategy that can enhance the performance with a tight memory budget. Thorough empirical studies demonstrate that, by tackling the memory explosion problem and incorporating topological information into memory replay, PDGNNs with TEM significantly outperform state-of-the-art techniques, especially in the challenging class-incremental setting.","url_abs":"https://arxiv.org/abs/2401.13200v3","url_pdf":"https://arxiv.org/pdf/2401.13200v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"topology-aware-embedding-memory-for-learning","repo_url":"https://github.com/imzhangxikun/pdgnns","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2401.13200","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.13200"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/imzhangxikun/pdgnns","reach":{"status":"ok"}}],"summary":{"ran":8,"unverified":2},"by_repo_kind":{"official":{"samples":10,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"d5b22dda37737526","entry":"AF","repo":"imzhangxikun/pdgnns","repo_kind":"official","path":"source/visualize.py","file_url":"https://github.com/imzhangxikun/pdgnns/blob/HEAD/source/visualize.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d5b22dda37737526"}},{"code_sha256_prefix":"96f655933698d722","entry":"AF_err","repo":"imzhangxikun/pdgnns","repo_kind":"official","path":"source/visualize.py","file_url":"https://github.com/imzhangxikun/pdgnns/blob/HEAD/source/visualize.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"96f655933698d722"}},{"code_sha256_prefix":"b2274beddc1f2c97","entry":"AP_err","repo":"imzhangxikun/pdgnns","repo_kind":"official","path":"source/visualize.py","file_url":"https://github.com/imzhangxikun/pdgnns/blob/HEAD/source/visualize.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b2274beddc1f2c97"}},{"code_sha256_prefix":"9e86bfded038ab51","entry":"accuracy","repo":"imzhangxikun/pdgnns","repo_kind":"official","path":"source/Backbones/utils.py","file_url":"https://github.com/imzhangxikun/pdgnns/blob/HEAD/source/Backbones/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9e86bfded038ab51"}},{"code_sha256_prefix":"dd247bc61b689793","entry":"compose_hyper_params","repo":"imzhangxikun/pdgnns","repo_kind":"official","path":"source/training/utils.py","file_url":"https://github.com/imzhangxikun/pdgnns/blob/HEAD/source/training/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dd247bc61b689793"}},{"code_sha256_prefix":"db9df3652df16af3","entry":"mask_init","repo":"imzhangxikun/pdgnns","repo_kind":"official","path":"source/Backbones/gnnconv.py","file_url":"https://github.com/imzhangxikun/pdgnns/blob/HEAD/source/Backbones/gnnconv.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"db9df3652df16af3"}},{"code_sha256_prefix":"02735936f54bba89","entry":"remove_illegal_characters","repo":"imzhangxikun/pdgnns","repo_kind":"official","path":"source/training/utils.py","file_url":"https://github.com/imzhangxikun/pdgnns/blob/HEAD/source/training/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"02735936f54bba89"}},{"code_sha256_prefix":"43e1878c5f74b9aa","entry":"str2dict","repo":"imzhangxikun/pdgnns","repo_kind":"official","path":"source/training/utils.py","file_url":"https://github.com/imzhangxikun/pdgnns/blob/HEAD/source/training/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"43e1878c5f74b9aa"}},{"code_sha256_prefix":"78ee86b31e2503da","entry":"MultiClassCrossEntropy","repo":"imzhangxikun/pdgnns","repo_kind":"official","path":"source/Baselines/New_NCGL_model.py","file_url":"https://github.com/imzhangxikun/pdgnns/blob/HEAD/source/Baselines/New_NCGL_model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"78ee86b31e2503da"}},{"code_sha256_prefix":"633ce83fcdc2ed0b","entry":"get_model","repo":"imzhangxikun/pdgnns","repo_kind":"official","path":"source/Backbones/model_factory.py","file_url":"https://github.com/imzhangxikun/pdgnns/blob/HEAD/source/Backbones/model_factory.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"633ce83fcdc2ed0b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}