{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/distributed-variational-inference-in-sparse","title":"Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models","arxiv_id":"1402.1389","date":"2014-02-06","proceeding":"NeurIPS 2014 12","authors":["Yarin Gal","Mark van der Wilk","Carl E. Rasmussen"],"abstract":"Gaussian processes (GPs) are a powerful tool for probabilistic inference over\nfunctions. They have been applied to both regression and non-linear\ndimensionality reduction, and offer desirable properties such as uncertainty\nestimates, robustness to over-fitting, and principled ways for tuning\nhyper-parameters. However the scalability of these models to big datasets\nremains an active topic of research. We introduce a novel re-parametrisation of\nvariational inference for sparse GP regression and latent variable models that\nallows for an efficient distributed algorithm. This is done by exploiting the\ndecoupling of the data given the inducing points to re-formulate the evidence\nlower bound in a Map-Reduce setting. We show that the inference scales well\nwith data and computational resources, while preserving a balanced distribution\nof the load among the nodes. We further demonstrate the utility in scaling\nGaussian processes to big data. We show that GP performance improves with\nincreasing amounts of data in regression (on flight data with 2 million\nrecords) and latent variable modelling (on MNIST). The results show that GPs\nperform better than many common models often used for big data.","url_abs":"http://arxiv.org/abs/1402.1389v2","url_pdf":"http://arxiv.org/pdf/1402.1389v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"distributed-variational-inference-in-sparse","repo_url":"https://github.com/markvdw/GParML","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"dimensionality-reduction","task_name":"Dimensionality Reduction"},{"task_slug":"gaussian-processes","task_name":"Gaussian Processes"},{"task_slug":"variational-inference","task_name":"Variational Inference"},{"task_slug":"regression-1","task_name":"regression"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1402.1389","atlas_url":"https://app.syntology.ai/?focus=1402.1389","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}