{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-better-out-of-distribution","title":"Towards Better Out-of-Distribution Generalization of Neural Algorithmic Reasoning Tasks","arxiv_id":"2211.00692","date":"2022-11-01","proceeding":null,"authors":["Sadegh Mahdavi","Kevin Swersky","Thomas Kipf","Milad Hashemi","Christos Thrampoulidis","Renjie Liao"],"abstract":"In this paper, we study the OOD generalization of neural algorithmic reasoning tasks, where the goal is to learn an algorithm (e.g., sorting, breadth-first search, and depth-first search) from input-output pairs using deep neural networks. First, we argue that OOD generalization in this setting is significantly different than common OOD settings. For example, some phenomena in OOD generalization of image classifications such as \\emph{accuracy on the line} are not observed here, and techniques such as data augmentation methods do not help as assumptions underlying many augmentation techniques are often violated. Second, we analyze the main challenges (e.g., input distribution shift, non-representative data generation, and uninformative validation metrics) of the current leading benchmark, i.e., CLRS \\citep{deepmind2021clrs}, which contains 30 algorithmic reasoning tasks. We propose several solutions, including a simple-yet-effective fix to the input distribution shift and improved data generation. Finally, we propose an attention-based 2WL-graph neural network (GNN) processor which complements message-passing GNNs so their combination outperforms the state-of-the-art model by a 3% margin averaged over all algorithms. Our code is available at: \\url{https://github.com/smahdavi4/clrs}.","url_abs":"https://arxiv.org/abs/2211.00692v2","url_pdf":"https://arxiv.org/pdf/2211.00692v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-better-out-of-distribution","repo_url":"https://github.com/smahdavi4/clrs","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"jax","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"graph-neural-network","task_name":"Graph Neural Network"},{"task_slug":"out-of-distribution-generalization","task_name":"Out-of-Distribution Generalization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2211.00692","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2211.00692"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/smahdavi4/clrs","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":2,"unverified":4},"by_repo_kind":{"official":{"samples":6,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"af9e8ccc380610b6","entry":"array","repo":"smahdavi4/clrs","repo_kind":"official","path":"clrs/_src/probing.py","file_url":"https://github.com/smahdavi4/clrs/blob/HEAD/clrs/_src/probing.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"af9e8ccc380610b6"}},{"code_sha256_prefix":"9cfc6b7e82435a99","entry":"opt_update","repo":"smahdavi4/clrs","repo_kind":"official","path":"clrs/_src/baselines.py","file_url":"https://github.com/smahdavi4/clrs/blob/HEAD/clrs/_src/baselines.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9cfc6b7e82435a99"}},{"code_sha256_prefix":"76a187450d32d861","entry":"filter_null_grads","repo":"smahdavi4/clrs","repo_kind":"official","path":"clrs/_src/baselines.py","file_url":"https://github.com/smahdavi4/clrs/blob/HEAD/clrs/_src/baselines.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"76a187450d32d861"}},{"code_sha256_prefix":"cc85a2f3114ddf9a","entry":"gumbel_softmax","repo":"smahdavi4/clrs","repo_kind":"official","path":"clrs/_src/nets.py","file_url":"https://github.com/smahdavi4/clrs/blob/HEAD/clrs/_src/nets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cc85a2f3114ddf9a"}},{"code_sha256_prefix":"e8e97aca8a38a6c4","entry":"straight_through_sigmoid","repo":"smahdavi4/clrs","repo_kind":"official","path":"clrs/_src/nets.py","file_url":"https://github.com/smahdavi4/clrs/blob/HEAD/clrs/_src/nets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e8e97aca8a38a6c4"}},{"code_sha256_prefix":"d54a58a511f318fd","entry":"vq_quantize","repo":"smahdavi4/clrs","repo_kind":"official","path":"clrs/_src/nets.py","file_url":"https://github.com/smahdavi4/clrs/blob/HEAD/clrs/_src/nets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d54a58a511f318fd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}