{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/comparing-graph-transformers-via-positional","title":"Comparing Graph Transformers via Positional Encodings","arxiv_id":"2402.14202","date":"2024-02-22","proceeding":null,"authors":["Mitchell Black","Zhengchao Wan","Gal Mishne","Amir Nayyeri","Yusu Wang"],"abstract":"The distinguishing power of graph transformers is closely tied to the choice of positional encoding: features used to augment the base transformer with information about the graph. There are two primary types of positional encoding: absolute positional encodings (APEs) and relative positional encodings (RPEs). APEs assign features to each node and are given as input to the transformer. RPEs instead assign a feature to each pair of nodes, e.g., graph distance, and are used to augment the attention block. A priori, it is unclear which method is better for maximizing the power of the resulting graph transformer. In this paper, we aim to understand the relationship between these different types of positional encodings. Interestingly, we show that graph transformers using APEs and RPEs are equivalent in terms of distinguishing power. In particular, we demonstrate how to interchange APEs and RPEs while maintaining their distinguishing power in terms of graph transformers. Based on our theoretical results, we provide a study on several APEs and RPEs (including the resistance distance and the recently introduced stable and expressive positional encoding (SPE)) and compare their distinguishing power in terms of transformers. We believe our work will help navigate the huge number of choices of positional encoding and will provide guidance on the future design of positional encodings for graph transformers.","url_abs":"https://arxiv.org/abs/2402.14202v4","url_pdf":"https://arxiv.org/pdf/2402.14202v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"comparing-graph-transformers-via-positional","repo_url":"https://github.com/blackmit/comparing_graph_transformers_via_positional_encodings","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"navigate","task_name":"Navigate"}],"methods":[{"method_slug":"base","method_name":"BASE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2402.14202","atlas_url":"https://app.syntology.ai/?focus=2402.14202","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.14202"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/blackmit/comparing_graph_transformers_via_positional_encodings","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"unverified":6},"by_repo_kind":{"official":{"samples":8,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1bb331bf70a78136","entry":"get_final_pretrained_ckpt","repo":"blackmit/comparing_graph_transformers_via_positional_encodings","repo_kind":"official","path":"graphgps/finetuning.py","file_url":"https://github.com/blackmit/comparing_graph_transformers_via_positional_encodings/blob/HEAD/graphgps/finetuning.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1bb331bf70a78136"}},{"code_sha256_prefix":"55e0e3e0c19f368e","entry":"graph_token_pooling","repo":"blackmit/comparing_graph_transformers_via_positional_encodings","repo_kind":"official","path":"graphgps/pooling/graph_token.py","file_url":"https://github.com/blackmit/comparing_graph_transformers_via_positional_encodings/blob/HEAD/graphgps/pooling/graph_token.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"55e0e3e0c19f368e"}},{"code_sha256_prefix":"ae56bcfaae96da3c","entry":"apply_chunking_to_forward","repo":"blackmit/comparing_graph_transformers_via_positional_encodings","repo_kind":"official","path":"graphgps/layer/bigbird_layer.py","file_url":"https://github.com/blackmit/comparing_graph_transformers_via_positional_encodings/blob/HEAD/graphgps/layer/bigbird_layer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ae56bcfaae96da3c"}},{"code_sha256_prefix":"3db4474abe13fdee","entry":"init_model_from_pretrained","repo":"blackmit/comparing_graph_transformers_via_positional_encodings","repo_kind":"official","path":"graphgps/finetuning.py","file_url":"https://github.com/blackmit/comparing_graph_transformers_via_positional_encodings/blob/HEAD/graphgps/finetuning.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3db4474abe13fdee"}},{"code_sha256_prefix":"c98a702be0660ac9","entry":"is_seed","repo":"blackmit/comparing_graph_transformers_via_positional_encodings","repo_kind":"official","path":"graphgps/agg_runs.py","file_url":"https://github.com/blackmit/comparing_graph_transformers_via_positional_encodings/blob/HEAD/graphgps/agg_runs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c98a702be0660ac9"}},{"code_sha256_prefix":"438635c08adfe2aa","entry":"is_split","repo":"blackmit/comparing_graph_transformers_via_positional_encodings","repo_kind":"official","path":"graphgps/agg_runs.py","file_url":"https://github.com/blackmit/comparing_graph_transformers_via_positional_encodings/blob/HEAD/graphgps/agg_runs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"438635c08adfe2aa"}},{"code_sha256_prefix":"40e98e1ecbf39f34","entry":"join_list","repo":"blackmit/comparing_graph_transformers_via_positional_encodings","repo_kind":"official","path":"graphgps/agg_runs.py","file_url":"https://github.com/blackmit/comparing_graph_transformers_via_positional_encodings/blob/HEAD/graphgps/agg_runs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"40e98e1ecbf39f34"}},{"code_sha256_prefix":"d796cdea5889c847","entry":"load_pretrained_model_cfg","repo":"blackmit/comparing_graph_transformers_via_positional_encodings","repo_kind":"official","path":"graphgps/finetuning.py","file_url":"https://github.com/blackmit/comparing_graph_transformers_via_positional_encodings/blob/HEAD/graphgps/finetuning.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d796cdea5889c847"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}