Papers › New Benchmarks for Learning on Non-Homophilous Graphs

New Benchmarks for Learning on Non-Homophilous Graphs

3 Apr 2021arXiv:2104.01404archive 2025-07-28

Derek Lim, Xiuyu Li, Felix Hohne, Ser-Nam Lim

Much data with graph structures satisfy the principle of homophily, meaning that connected nodes tend to be similar with respect to a specific attribute. As such, ubiquitous datasets for graph machine learning tasks have generally been highly homophilous, rewarding methods that leverage homophily as an inductive bias. Recent work has pointed out this particular focus, as new non-homophilous datasets have been introduced and graph representation learning models better suited for low-homophily settings have been developed. However, these datasets are small and poorly suited to truly testing the effectiveness of new methods in non-homophilous settings. We present a series of improved graph datasets with node label relationships that do not satisfy the homophily principle. Along with this, we introduce a new measure of the presence or absence of homophily that is better suited than existing measures in different regimes. We benchmark a range of simple methods and graph neural networks across our proposed datasets, drawing new insights for further research. Data and codes can be found at https://github.com/CUAI/Non-Homophily-Benchmarks.

PaperPDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

CUAI/Non-Homophily-Benchmarks officialmentioned in paperpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods (arXiv:2110.14446; that paper's own run record: 15 ran (of which 5 constructed an object rather than computing a result; 10 with no instrument failure: 1 honoured, 0 violated, 9 with no contract checked; 5 where Syntology's instrument failed) · 6 unverified); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

AttributeFraud DetectionGraph Representation LearningInductive BiasNode ClassificationNode Classification on Non-Homophilic (Heterophilic) GraphsRepresentation Learning

Datasets

Introduced by this paper, per the archive.

Deezer-Europe

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Fraud Detection Yelp-Fraud GAT+JK AUC-ROC 90.04 #6 of 10 Archive leaderboard report
Node Classification Penn94 GCNJK Accuracy 81.63 ± 0.54 #18 of 32 Archive leaderboard report
Node Classification Penn94 LINK Accuracy 80.79 ± 0.49 #22 of 32 Archive leaderboard report
Node Classification Penn94 GATJK Accuracy 80.69 ± 0.36 #23 of 32 Archive leaderboard report
Node Classification Penn94 L Prop 2-hop Accuracy 74.13 ± 0.46 #29 of 32 Archive leaderboard report
Node Classification Penn94 MLP Accuracy 73.61 ± 0.40 #30 of 32 Archive leaderboard report
Node Classification Penn94 L Prop 1-hop Accuracy 63.21 ± 0.39 #32 of 32 Archive leaderboard report
Node Classification Yelp-Fraud GAT+JK AUC-ROC 90.04 #5 of 9 Archive leaderboard report
Node Classification genius LINK Accuracy 73.56 ± 0.14 #21 of 26 Archive leaderboard report
Node Classification genius L Prop 2-hop Accuracy 67.04 ± 0.20 #22 of 26 Archive leaderboard report
Node Classification genius L Prop 1-hop Accuracy 66.02 ± 0.16 #23 of 26 Archive leaderboard report
Node Classification genius GATJK Accuracy 56.70 ± 2.07 #24 of 26 Archive leaderboard report
Node Classification on Non-Homophilic (Heterophilic) Graphs Deezer-Europe MLP-2 1:1 Accuracy 66.55±0.72 #14 of 28 Archive leaderboard report
Node Classification on Non-Homophilic (Heterophilic) Graphs Deezer-Europe GCN+JK 1:1 Accuracy 60.99±0.14 #23 of 28 Archive leaderboard report
Node Classification on Non-Homophilic (Heterophilic) Graphs Deezer-Europe GAT+JK 1:1 Accuracy 59.66±0.92 #25 of 28 Archive leaderboard report
Node Classification on Non-Homophilic (Heterophilic) Graphs Deezer-Europe LINK 1:1 Accuracy 57.71±0.36 #26 of 28 Archive leaderboard report
Node Classification on Non-Homophilic (Heterophilic) Graphs Deezer-Europe LProp (2hop) 1:1 Accuracy 56.96±0.26 #27 of 28 Archive leaderboard report
Node Classification on Non-Homophilic (Heterophilic) Graphs Deezer-Europe L Prop (1hop) 1:1 Accuracy 56.50±0.41 #28 of 28 Archive leaderboard report
Node Classification on Non-Homophilic (Heterophilic) Graphs Wisconsin(60%/20%/20% random splits) MLP-2 1:1 Accuracy 93.87 ± 3.33 #14 of 32 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections