{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/improving-attention-mechanism-in-graph-neural","title":"Improving Attention Mechanism in Graph Neural Networks via Cardinality Preservation","arxiv_id":"1907.02204","date":"2019-07-04","proceeding":null,"authors":["Shuo Zhang","Lei Xie"],"abstract":"Graph Neural Networks (GNNs) are powerful to learn the representation of graph-structured data. Most of the GNNs use the message-passing scheme, where the embedding of a node is iteratively updated by aggregating the information of its neighbors. To achieve a better expressive capability of node influences, attention mechanism has grown to be popular to assign trainable weights to the nodes in aggregation. Though the attention-based GNNs have achieved remarkable results in various tasks, a clear understanding of their discriminative capacities is missing. In this work, we present a theoretical analysis of the representational properties of the GNN that adopts the attention mechanism as an aggregator. Our analysis determines all cases when those attention-based GNNs can always fail to distinguish certain distinct structures. Those cases appear due to the ignorance of cardinality information in attention-based aggregation. To improve the performance of attention-based GNNs, we propose cardinality preserved attention (CPA) models that can be applied to any kind of attention mechanisms. Our experiments on node and graph classification confirm our theoretical analysis and show the competitive performance of our CPA models.","url_abs":"https://arxiv.org/abs/1907.02204v4","url_pdf":"https://arxiv.org/pdf/1907.02204v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"improving-attention-mechanism-in-graph-neural","repo_url":"https://github.com/zetayue/CPA","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"graph-classification","task_name":"Graph Classification"},{"task_slug":"graph-representation-learning","task_name":"Graph Representation Learning"},{"task_slug":"node-classification","task_name":"Node Classification"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/graph-classification-on-enzymes","task":"Graph Classification","dataset":"ENZYMES","model":"GAT-GC (f-Scaled)","rank_in_archive_order":32,"of":54,"metrics":{"Accuracy":"58.45"},"uses_additional_data":false},{"leaderboard":"/sota/graph-classification-on-mutag","task":"Graph Classification","dataset":"MUTAG","model":"GAT-GC (f-Scaled)","rank_in_archive_order":20,"of":74,"metrics":{"Accuracy":"90.44%"},"uses_additional_data":true},{"leaderboard":"/sota/graph-classification-on-nci1","task":"Graph Classification","dataset":"NCI1","model":"GAT-GC (f-Scaled)","rank_in_archive_order":32,"of":69,"metrics":{"Accuracy":"82.28%"},"uses_additional_data":false},{"leaderboard":"/sota/graph-classification-on-proteins","task":"Graph Classification","dataset":"PROTEINS","model":"GAT-GC (f-Scaled)","rank_in_archive_order":38,"of":103,"metrics":{"Accuracy":"76.81%"},"uses_additional_data":false},{"leaderboard":"/sota/graph-classification-on-re-m5k","task":"Graph Classification","dataset":"RE-M5K","model":"GAT-GC (f-Scaled)","rank_in_archive_order":2,"of":8,"metrics":{"Accuracy":"57.22%"},"uses_additional_data":false},{"leaderboard":"/sota/graph-classification-on-reddit-b","task":"Graph Classification","dataset":"REDDIT-B","model":"GAT-GC (f-Scaled)","rank_in_archive_order":2,"of":12,"metrics":{"Accuracy":"92.57"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1907.02204","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1907.02204"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zetayue/CPA","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":1},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"881756aa4bdfbacb","entry":"separate_data","repo":"zetayue/CPA","repo_kind":"official","path":"util.py","file_url":"https://github.com/zetayue/CPA/blob/HEAD/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"881756aa4bdfbacb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}