{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/graph-robustness-benchmark-benchmarking-the","title":"Graph Robustness Benchmark: Benchmarking the Adversarial Robustness of Graph Machine Learning","arxiv_id":"2111.04314","date":"2021-11-08","proceeding":null,"authors":["Qinkai Zheng","Xu Zou","Yuxiao Dong","Yukuo Cen","Da Yin","Jiarong Xu","Yang Yang","Jie Tang"],"abstract":"Adversarial attacks on graphs have posed a major threat to the robustness of graph machine learning (GML) models. Naturally, there is an ever-escalating arms race between attackers and defenders. However, the strategies behind both sides are often not fairly compared under the same and realistic conditions. To bridge this gap, we present the Graph Robustness Benchmark (GRB) with the goal of providing a scalable, unified, modular, and reproducible evaluation for the adversarial robustness of GML models. GRB standardizes the process of attacks and defenses by 1) developing scalable and diverse datasets, 2) modularizing the attack and defense implementations, and 3) unifying the evaluation protocol in refined scenarios. By leveraging the GRB pipeline, the end-users can focus on the development of robust GML models with automated data processing and experimental evaluations. To support open and reproducible research on graph adversarial learning, GRB also hosts public leaderboards across different scenarios. As a starting point, we conduct extensive experiments to benchmark baseline techniques. GRB is open-source and welcomes contributions from the community. Datasets, codes, leaderboards are available at https://cogdl.ai/grb/home.","url_abs":"https://arxiv.org/abs/2111.04314v1","url_pdf":"https://arxiv.org/pdf/2111.04314v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"graph-robustness-benchmark-benchmarking-the","repo_url":"https://github.com/thudm/grb","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"adversarial-robustness","task_name":"Adversarial Robustness"},{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"},{"task_slug":"benchmarking","task_name":"Benchmarking"}],"methods":[],"datasets_introduced":[{"slug":"grb","name":"GRB","full_name":"Graph Robustness Benchmark"}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2111.04314","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2111.04314"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/THUDM/grb","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thudm/grb","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3,"unverified":2},"by_repo_kind":{"official":{"samples":5,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"399418345f3feb4b","entry":"eval_acc","repo":"THUDM/grb","repo_kind":"official","path":"grb/evaluator/metric.py","file_url":"https://github.com/THUDM/grb/blob/HEAD/grb/evaluator/metric.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"399418345f3feb4b"}},{"code_sha256_prefix":"cfa29e2de3f6b394","entry":"eval_f1multilabel","repo":"THUDM/grb","repo_kind":"official","path":"grb/evaluator/metric.py","file_url":"https://github.com/THUDM/grb/blob/HEAD/grb/evaluator/metric.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cfa29e2de3f6b394"}},{"code_sha256_prefix":"4eae1658e4eeb3f7","entry":"eval_rocauc","repo":"THUDM/grb","repo_kind":"official","path":"grb/evaluator/metric.py","file_url":"https://github.com/THUDM/grb/blob/HEAD/grb/evaluator/metric.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4eae1658e4eeb3f7"}},{"code_sha256_prefix":"bdd3cd5c0cdc99df","entry":"feat_normalize","repo":"THUDM/grb","repo_kind":"official","path":"grb/dataset/dataset.py","file_url":"https://github.com/THUDM/grb/blob/HEAD/grb/dataset/dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bdd3cd5c0cdc99df"}},{"code_sha256_prefix":"854c93cc02b05a8a","entry":"splitting","repo":"THUDM/grb","repo_kind":"official","path":"grb/dataset/dataset.py","file_url":"https://github.com/THUDM/grb/blob/HEAD/grb/dataset/dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"854c93cc02b05a8a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}