{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/accurate-learning-of-graph-representations-1","title":"Accurate Learning of Graph Representations with Graph Multiset Pooling","arxiv_id":"2102.11533","date":"2021-02-23","proceeding":"ICLR 2021 1","authors":["Jinheon Baek","Minki Kang","Sung Ju Hwang"],"abstract":"Graph neural networks have been widely used on modeling graph data, achieving impressive results on node classification and link prediction tasks. Yet, obtaining an accurate representation for a graph further requires a pooling function that maps a set of node representations into a compact form. A simple sum or average over all node representations considers all node features equally without consideration of their task relevance, and any structural dependencies among them. Recently proposed hierarchical graph pooling methods, on the other hand, may yield the same representation for two different graphs that are distinguished by the Weisfeiler-Lehman test, as they suboptimally preserve information from the node features. To tackle these limitations of existing graph pooling methods, we first formulate the graph pooling problem as a multiset encoding problem with auxiliary information about the graph structure, and propose a Graph Multiset Transformer (GMT) which is a multi-head attention based global pooling layer that captures the interaction between nodes according to their structural dependencies. We show that GMT satisfies both injectiveness and permutation invariance, such that it is at most as powerful as the Weisfeiler-Lehman graph isomorphism test. Moreover, our methods can be easily extended to the previous node clustering approaches for hierarchical graph pooling. Our experimental results show that GMT significantly outperforms state-of-the-art graph pooling methods on graph classification benchmarks with high memory and time efficiency, and obtains even larger performance gain on graph reconstruction and generation tasks.","url_abs":"https://arxiv.org/abs/2102.11533v4","url_pdf":"https://arxiv.org/pdf/2102.11533v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"accurate-learning-of-graph-representations-1","repo_url":"https://github.com/JinheonBaek/GMT","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"graph-classification","task_name":"Graph Classification"},{"task_slug":"graph-clustering","task_name":"Graph Clustering"},{"task_slug":"graph-embedding","task_name":"Graph Embedding"},{"task_slug":"graph-generation","task_name":"Graph Generation"},{"task_slug":"graph-learning","task_name":"Graph Learning"},{"task_slug":"graph-reconstruction","task_name":"Graph Reconstruction"},{"task_slug":"node-clustering","task_name":"Node Clustering"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/graph-classification-on-bbbp","task":"Graph Classification","dataset":"BBBP","model":"GMT","rank_in_archive_order":3,"of":3,"metrics":{"ROC-AUC":"68.31"},"uses_additional_data":false},{"leaderboard":"/sota/graph-classification-on-collab","task":"Graph Classification","dataset":"COLLAB","model":"GMT","rank_in_archive_order":9,"of":39,"metrics":{"Accuracy":"80.74%"},"uses_additional_data":false},{"leaderboard":"/sota/graph-classification-on-dd","task":"Graph Classification","dataset":"D&D","model":"GMT","rank_in_archive_order":23,"of":53,"metrics":{"Accuracy":"78.72%"},"uses_additional_data":false},{"leaderboard":"/sota/graph-classification-on-hiv","task":"Graph Classification","dataset":"HIV","model":"GMT","rank_in_archive_order":2,"of":3,"metrics":{"ROC-AUC":"77.56"},"uses_additional_data":false},{"leaderboard":"/sota/graph-classification-on-hiv-dataset","task":"Graph Classification","dataset":"HIV dataset","model":"GMT","rank_in_archive_order":3,"of":5,"metrics":{"ROC-AUC":"77.56"},"uses_additional_data":false},{"leaderboard":"/sota/graph-classification-on-imdb-b","task":"Graph Classification","dataset":"IMDb-B","model":"GMT","rank_in_archive_order":31,"of":51,"metrics":{"Accuracy":"73.48%"},"uses_additional_data":false},{"leaderboard":"/sota/graph-classification-on-imdb-m","task":"Graph Classification","dataset":"IMDb-M","model":"GMT","rank_in_archive_order":19,"of":36,"metrics":{"Accuracy":"50.66%"},"uses_additional_data":false},{"leaderboard":"/sota/graph-classification-on-mutag","task":"Graph Classification","dataset":"MUTAG","model":"GMT","rank_in_archive_order":66,"of":74,"metrics":{"Accuracy":"83.44%"},"uses_additional_data":false},{"leaderboard":"/sota/graph-classification-on-proteins","task":"Graph Classification","dataset":"PROTEINS","model":"GMT","rank_in_archive_order":71,"of":103,"metrics":{"Accuracy":"75.09%"},"uses_additional_data":false},{"leaderboard":"/sota/graph-classification-on-tox21","task":"Graph Classification","dataset":"Tox21","model":"GMT","rank_in_archive_order":1,"of":3,"metrics":{"ROC-AUC":"77.3"},"uses_additional_data":false},{"leaderboard":"/sota/graph-classification-on-toxcast","task":"Graph Classification","dataset":"ToxCast","model":"GMT","rank_in_archive_order":1,"of":3,"metrics":{"ROC-AUC":"65.44"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2102.11533","atlas_url":"https://app.syntology.ai/?focus=2102.11533","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2102.11533"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/JinheonBaek/GMT","reach":null}],"summary":{"ran":4,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"63d1824fb4841fdd","entry":"GraphRepresentation","repo":"JinheonBaek/GMT","repo_kind":"official","path":"models/nets.py","file_url":"https://github.com/JinheonBaek/GMT/blob/HEAD/models/nets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"63d1824fb4841fdd"}},{"code_sha256_prefix":"0060b5109c3cec3e","entry":"MAB","repo":"JinheonBaek/GMT","repo_kind":"official","path":"models/nets.py","file_url":"https://github.com/JinheonBaek/GMT/blob/HEAD/models/nets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0060b5109c3cec3e"}},{"code_sha256_prefix":"44973cac2b91988d","entry":"PMA","repo":"JinheonBaek/GMT","repo_kind":"official","path":"models/nets.py","file_url":"https://github.com/JinheonBaek/GMT/blob/HEAD/models/nets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"44973cac2b91988d"}},{"code_sha256_prefix":"4bf5f02b2a1a85d8","entry":"SAB","repo":"JinheonBaek/GMT","repo_kind":"official","path":"models/nets.py","file_url":"https://github.com/JinheonBaek/GMT/blob/HEAD/models/nets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4bf5f02b2a1a85d8"}},{"code_sha256_prefix":"758394cc1690586d","entry":"GraphMultisetTransformer","repo":"JinheonBaek/GMT","repo_kind":"official","path":"models/nets.py","file_url":"https://github.com/JinheonBaek/GMT/blob/HEAD/models/nets.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"758394cc1690586d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}