{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/near-optimal-approximations-for-bayesian","title":"Near-Optimal Approximations for Bayesian Inference in Function Space","arxiv_id":"2502.18279","date":"2025-02-25","proceeding":null,"authors":["Veit Wild","James Wu","Dino Sejdinovic","Jeremias Knoblauch"],"abstract":"We propose a scalable inference algorithm for Bayes posteriors defined on a reproducing kernel Hilbert space (RKHS). Given a likelihood function and a Gaussian random element representing the prior, the corresponding Bayes posterior measure $\\Pi_{\\text{B}}$ can be obtained as the stationary distribution of an RKHS-valued Langevin diffusion. We approximate the infinite-dimensional Langevin diffusion via a projection onto the first $M$ components of the Kosambi-Karhunen-Lo\\`eve expansion. Exploiting the thus obtained approximate posterior for these $M$ components, we perform inference for $\\Pi_{\\text{B}}$ by relying on the law of total probability and a sufficiency assumption. The resulting method scales as $O(M^3+JM^2)$, where $J$ is the number of samples produced from the posterior measure $\\Pi_{\\text{B}}$. Interestingly, the algorithm recovers the posterior arising from the sparse variational Gaussian process (SVGP) (see Titsias, 2009) as a special case, owed to the fact that the sufficiency assumption underlies both methods. However, whereas the SVGP is parametrically constrained to be a Gaussian process, our method is based on a non-parametric variational family $\\mathcal{P}(\\mathbb{R}^M)$ consisting of all probability measures on $\\mathbb{R}^M$. As a result, our method is provably close to the optimal $M$-dimensional variational approximation of the Bayes posterior $\\Pi_{\\text{B}}$ in $\\mathcal{P}(\\mathbb{R}^M)$ for convex and Lipschitz continuous negative log likelihoods, and coincides with SVGP for the special case of a Gaussian error likelihood.","url_abs":"https://arxiv.org/abs/2502.18279v1","url_pdf":"https://arxiv.org/pdf/2502.18279v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"near-optimal-approximations-for-bayesian","repo_url":"https://github.com/jswu18/projected-langevin-sampling","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"bayesian-inference","task_name":"Bayesian Inference"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"gaussian-process","method_name":"Gaussian Process"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}