{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/attentive-neural-processes","title":"Attentive Neural Processes","arxiv_id":"1901.05761","date":"2019-01-17","proceeding":"ICLR 2019 5","authors":["Hyunjik Kim","andriy mnih","Jonathan Schwarz","Marta Garnelo","Ali Eslami","Dan Rosenbaum","Oriol Vinyals","Yee Whye Teh"],"abstract":"Neural Processes (NPs) (Garnelo et al 2018a;b) approach regression by learning to map a context set of observed input-output pairs to a distribution over regression functions. Each function models the distribution of the output given an input, conditioned on the context. NPs have the benefit of fitting observed data efficiently with linear complexity in the number of context input-output pairs, and can learn a wide family of conditional distributions; they learn predictive distributions conditioned on context sets of arbitrary size. Nonetheless, we show that NPs suffer a fundamental drawback of underfitting, giving inaccurate predictions at the inputs of the observed data they condition on. We address this issue by incorporating attention into NPs, allowing each input location to attend to the relevant context points for the prediction. We show that this greatly improves the accuracy of predictions, results in noticeably faster training, and expands the range of functions that can be modelled.","url_abs":"https://arxiv.org/abs/1901.05761v2","url_pdf":"https://arxiv.org/pdf/1901.05761v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"attentive-neural-processes","repo_url":"https://github.com/deepmind/neural-processes","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"attentive-neural-processes","repo_url":"https://github.com/3springs/np_vs_kriging","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"attentive-neural-processes","repo_url":"https://github.com/MIT-Omnipush/omnipush-metalearning-baselines","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"attentive-neural-processes","repo_url":"https://github.com/deepmind/conditional-neural-process","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"attentive-neural-processes","repo_url":"https://github.com/google-deepmind/neural-processes","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"attentive-neural-processes","repo_url":"https://github.com/revsic/tf-neural-process","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"attentive-neural-processes","repo_url":"https://github.com/soobinseo/attentive-neural-process","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"attentive-neural-processes","repo_url":"https://github.com/wyn430/NP_ODE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"regression-1","task_name":"regression"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1901.05761","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1901.05761"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/revsic/tf-neural-process","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/3springs/np_vs_kriging","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/deepmind/neural-processes","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-deepmind/neural-processes","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wyn430/NP_ODE","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MIT-Omnipush/omnipush-metalearning-baselines","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/deepmind/conditional-neural-process","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/soobinseo/attentive-neural-process","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":13},"by_repo_kind":{"listed":{"samples":13,"ran":0,"repositories":4}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d4a70fdf79805f9e","entry":"add_time","repo":"wyn430/NP_ODE","repo_kind":"listed","path":"NP_ODE/Decoder_ODE.py","file_url":"https://github.com/wyn430/NP_ODE/blob/HEAD/NP_ODE/Decoder_ODE.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d4a70fdf79805f9e"}},{"code_sha256_prefix":"c66bed3bb5416312","entry":"check_in_list","repo":"MIT-Omnipush/omnipush-metalearning-baselines","repo_kind":"listed","path":"preprocess.py","file_url":"https://github.com/MIT-Omnipush/omnipush-metalearning-baselines/blob/HEAD/preprocess.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c66bed3bb5416312"}},{"code_sha256_prefix":"598249d285dd3da3","entry":"collate_fn","repo":"soobinseo/attentive-neural-process","repo_kind":"listed","path":"preprocess.py","file_url":"https://github.com/soobinseo/attentive-neural-process/blob/HEAD/preprocess.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"598249d285dd3da3"}},{"code_sha256_prefix":"d89f70014190873d","entry":"conv3x3","repo":"wyn430/NP_ODE","repo_kind":"listed","path":"NP_ODE/Decoder_ODE.py","file_url":"https://github.com/wyn430/NP_ODE/blob/HEAD/NP_ODE/Decoder_ODE.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d89f70014190873d"}},{"code_sha256_prefix":"5ec581fc5c14068d","entry":"dense_sequential","repo":"revsic/tf-neural-process","repo_kind":"listed","path":"neural_process/module/tfutils.py","file_url":"https://github.com/revsic/tf-neural-process/blob/HEAD/neural_process/module/tfutils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5ec581fc5c14068d"}},{"code_sha256_prefix":"e961d0763aac6af6","entry":"dotprod_attention","repo":"revsic/tf-neural-process","repo_kind":"listed","path":"neural_process/module/attention.py","file_url":"https://github.com/revsic/tf-neural-process/blob/HEAD/neural_process/module/attention.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e961d0763aac6af6"}},{"code_sha256_prefix":"e5d5284f5dcd795e","entry":"laplace_attention","repo":"revsic/tf-neural-process","repo_kind":"listed","path":"neural_process/module/attention.py","file_url":"https://github.com/revsic/tf-neural-process/blob/HEAD/neural_process/module/attention.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e5d5284f5dcd795e"}},{"code_sha256_prefix":"7b7fa330431d3cae","entry":"norm","repo":"wyn430/NP_ODE","repo_kind":"listed","path":"NP_ODE/Decoder_ODE.py","file_url":"https://github.com/wyn430/NP_ODE/blob/HEAD/NP_ODE/Decoder_ODE.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7b7fa330431d3cae"}},{"code_sha256_prefix":"6e5c482aee3281e6","entry":"ode_solve","repo":"wyn430/NP_ODE","repo_kind":"listed","path":"NP_ODE/Neural_ODE.py","file_url":"https://github.com/wyn430/NP_ODE/blob/HEAD/NP_ODE/Neural_ODE.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6e5c482aee3281e6"}},{"code_sha256_prefix":"f13efa2dfc5c08cd","entry":"omnipush","repo":"MIT-Omnipush/omnipush-metalearning-baselines","repo_kind":"listed","path":"preprocess.py","file_url":"https://github.com/MIT-Omnipush/omnipush-metalearning-baselines/blob/HEAD/preprocess.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f13efa2dfc5c08cd"}},{"code_sha256_prefix":"dd6053b19e3b6819","entry":"omnipush_collate_fn","repo":"MIT-Omnipush/omnipush-metalearning-baselines","repo_kind":"listed","path":"preprocess.py","file_url":"https://github.com/MIT-Omnipush/omnipush-metalearning-baselines/blob/HEAD/preprocess.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dd6053b19e3b6819"}},{"code_sha256_prefix":"fb7e90554ec3877b","entry":"plot_functions","repo":"wyn430/NP_ODE","repo_kind":"listed","path":"NP_ODE/tools.py","file_url":"https://github.com/wyn430/NP_ODE/blob/HEAD/NP_ODE/tools.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fb7e90554ec3877b"}},{"code_sha256_prefix":"99de32874176bade","entry":"uniform_attention","repo":"revsic/tf-neural-process","repo_kind":"listed","path":"neural_process/module/attention.py","file_url":"https://github.com/revsic/tf-neural-process/blob/HEAD/neural_process/module/attention.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"99de32874176bade"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}