{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/universal-boosting-variational-inference","title":"Universal Boosting Variational Inference","arxiv_id":"1906.01235","date":"2019-06-04","proceeding":"NeurIPS 2019 12","authors":["Trevor Campbell","Xinglong Li"],"abstract":"Boosting variational inference (BVI) approximates an intractable probability density by iteratively building up a mixture of simple component distributions one at a time, using techniques from sparse convex optimization to provide both computational scalability and approximation error guarantees. But the guarantees have strong conditions that do not often hold in practice, resulting in degenerate component optimization problems; and we show that the ad-hoc regularization used to prevent degeneracy in practice can cause BVI to fail in unintuitive ways. We thus develop universal boosting variational inference (UBVI), a BVI scheme that exploits the simple geometry of probability densities under the Hellinger metric to prevent the degeneracy of other gradient-based BVI methods, avoid difficult joint optimizations of both component and weight, and simplify fully-corrective weight optimizations. We show that for any target density and any mixture component family, the output of UBVI converges to the best possible approximation in the mixture family, even when the mixture family is misspecified. We develop a scalable implementation based on exponential family mixture components and standard stochastic optimization techniques. Finally, we discuss statistical benefits of the Hellinger distance as a variational objective through bounds on posterior probability, moment, and importance sampling errors. Experiments on multiple datasets and models show that UBVI provides reliable, accurate posterior approximations.","url_abs":"https://arxiv.org/abs/1906.01235v2","url_pdf":"https://arxiv.org/pdf/1906.01235v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"universal-boosting-variational-inference","repo_url":"https://github.com/trevorcampbell/ubvi","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"universal-boosting-variational-inference","repo_url":"https://github.com/k-sobolev/UBVI","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"stochastic-optimization","task_name":"Stochastic Optimization"},{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1906.01235","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1906.01235"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/k-sobolev/UBVI","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/trevorcampbell/ubvi","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1a27b1d51a1493fc","entry":"adam","repo":"trevorcampbell/ubvi","repo_kind":"official","path":"ubvi/optimization/adam.py","file_url":"https://github.com/trevorcampbell/ubvi/blob/HEAD/ubvi/optimization/adam.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1a27b1d51a1493fc"}},{"code_sha256_prefix":"a75ac3c7c9b4db06","entry":"make_grad_logsumexp","repo":"trevorcampbell/ubvi","repo_kind":"official","path":"ubvi/autograd/logsumexp.py","file_url":"https://github.com/trevorcampbell/ubvi/blob/HEAD/ubvi/autograd/logsumexp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a75ac3c7c9b4db06"}},{"code_sha256_prefix":"a7c7ac335477a96c","entry":"sgd","repo":"trevorcampbell/ubvi","repo_kind":"official","path":"ubvi/optimization/sgd.py","file_url":"https://github.com/trevorcampbell/ubvi/blob/HEAD/ubvi/optimization/sgd.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a7c7ac335477a96c"}},{"code_sha256_prefix":"153dc5820a186816","entry":"simplex_sgd","repo":"trevorcampbell/ubvi","repo_kind":"official","path":"ubvi/optimization/simplex_sgd.py","file_url":"https://github.com/trevorcampbell/ubvi/blob/HEAD/ubvi/optimization/simplex_sgd.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"153dc5820a186816"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}