{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-signal-to-noise-ratio-issues-in","title":"On Signal-to-Noise Ratio Issues in Variational Inference for Deep Gaussian Processes","arxiv_id":"2011.00515","date":"2020-11-01","proceeding":null,"authors":["Tim G. J. Rudner","Oscar Key","Yarin Gal","Tom Rainforth"],"abstract":"We show that the gradient estimates used in training Deep Gaussian Processes (DGPs) with importance-weighted variational inference are susceptible to signal-to-noise ratio (SNR) issues. Specifically, we show both theoretically and via an extensive empirical evaluation that the SNR of the gradient estimates for the latent variable's variational parameters decreases as the number of importance samples increases. As a result, these gradient estimates degrade to pure noise if the number of importance samples is too large. To address this pathology, we show how doubly reparameterized gradient estimators, originally proposed for training variational autoencoders, can be adapted to the DGP setting and that the resultant estimators completely remedy the SNR issue, thereby providing more reliable training. Finally, we demonstrate that our fix can lead to consistent improvements in the predictive performance of DGP models.","url_abs":"https://arxiv.org/abs/2011.00515v2","url_pdf":"https://arxiv.org/pdf/2011.00515v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-signal-to-noise-ratio-issues-in","repo_url":"https://github.com/timrudner/snr_issues_in_deep_gps","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"gaussian-processes","task_name":"Gaussian Processes"},{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2011.00515","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2011.00515"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/hughsalimbeni/bayesian","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/timrudner/snr_issues_in_deep_gps","reach":null}],"summary":{"ran_honours":1,"ran_draft_wrong":2},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"49ab52bcc1f72492","entry":"get_k","repo":"timrudner/snr_issues_in_deep_gps","repo_kind":"official","path":"experiments/create_figures.py","file_url":"https://github.com/timrudner/snr_issues_in_deep_gps/blob/HEAD/experiments/create_figures.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"49ab52bcc1f72492"}},{"code_sha256_prefix":"c9c49c3ed975bca6","entry":"group_by_M","repo":"timrudner/snr_issues_in_deep_gps","repo_kind":"official","path":"experiments/create_figures.py","file_url":"https://github.com/timrudner/snr_issues_in_deep_gps/blob/HEAD/experiments/create_figures.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c9c49c3ed975bca6"}},{"code_sha256_prefix":"173c6baf6062e84d","entry":"load_gradients","repo":"timrudner/snr_issues_in_deep_gps","repo_kind":"official","path":"experiments/create_figures.py","file_url":"https://github.com/timrudner/snr_issues_in_deep_gps/blob/HEAD/experiments/create_figures.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"173c6baf6062e84d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}