{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multiple-importance-sampling-elbo-and-deep","title":"Multiple Importance Sampling ELBO and Deep Ensembles of Variational Approximations","arxiv_id":"2202.10951","date":"2022-02-22","proceeding":null,"authors":["Oskar Kviman","Harald Melin","Hazal Koptagel","Víctor Elvira","Jens Lagergren"],"abstract":"In variational inference (VI), the marginal log-likelihood is estimated using the standard evidence lower bound (ELBO), or improved versions as the importance weighted ELBO (IWELBO). We propose the multiple importance sampling ELBO (MISELBO), a \\textit{versatile} yet \\textit{simple} framework. MISELBO is applicable in both amortized and classical VI, and it uses ensembles, e.g., deep ensembles, of independently inferred variational approximations. As far as we are aware, the concept of deep ensembles in amortized VI has not previously been established. We prove that MISELBO provides a tighter bound than the average of standard ELBOs, and demonstrate empirically that it gives tighter bounds than the average of IWELBOs. MISELBO is evaluated in density-estimation experiments that include MNIST and several real-data phylogenetic tree inference problems. First, on the MNIST dataset, MISELBO boosts the density-estimation performances of a state-of-the-art model, nouveau VAE. Second, in the phylogenetic tree inference setting, our framework enhances a state-of-the-art VI algorithm that uses normalizing flows. On top of the technical benefits of MISELBO, it allows to unveil connections between VI and recent advances in the importance sampling literature, paving the way for further methodological advances. We provide our code at \\url{https://github.com/Lagergren-Lab/MISELBO}.","url_abs":"https://arxiv.org/abs/2202.10951v1","url_pdf":"https://arxiv.org/pdf/2202.10951v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multiple-importance-sampling-elbo-and-deep","repo_url":"https://github.com/lagergren-lab/miselbo","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"density-estimation","task_name":"Density Estimation"},{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[{"method_slug":"deep-ensembles","method_name":"Deep Ensembles"},{"method_slug":"variational-inference","method_name":"Variational Inference"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2202.10951","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2202.10951"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lagergren-lab/miselbo","reach":null}],"summary":{"ran_fixture":2,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"557137dfef22e3e0","entry":"compute_average_iwelbo","repo":"lagergren-lab/miselbo","repo_kind":"official","path":"miselbo_eval.py","file_url":"https://github.com/lagergren-lab/miselbo/blob/HEAD/miselbo_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"557137dfef22e3e0"}},{"code_sha256_prefix":"ad7a6459035e025a","entry":"compute_jsd","repo":"lagergren-lab/miselbo","repo_kind":"official","path":"miselbo_eval.py","file_url":"https://github.com/lagergren-lab/miselbo/blob/HEAD/miselbo_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ad7a6459035e025a"}},{"code_sha256_prefix":"6dfaafb487559e4b","entry":"compute_miselbo","repo":"lagergren-lab/miselbo","repo_kind":"official","path":"miselbo_eval.py","file_url":"https://github.com/lagergren-lab/miselbo/blob/HEAD/miselbo_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6dfaafb487559e4b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}