{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-mixture-learning-in-black-box","title":"Efficient Mixture Learning in Black-Box Variational Inference","arxiv_id":"2406.07083","date":"2024-06-11","proceeding":null,"authors":["Alexandra Hotti","Oskar Kviman","Ricky Molén","Víctor Elvira","Jens Lagergren"],"abstract":"Mixture variational distributions in black box variational inference (BBVI) have demonstrated impressive results in challenging density estimation tasks. However, currently scaling the number of mixture components can lead to a linear increase in the number of learnable parameters and a quadratic increase in inference time due to the evaluation of the evidence lower bound (ELBO). Our two key contributions address these limitations. First, we introduce the novel Multiple Importance Sampling Variational Autoencoder (MISVAE), which amortizes the mapping from input to mixture-parameter space using one-hot encodings. Fortunately, with MISVAE, each additional mixture component incurs a negligible increase in network parameters. Second, we construct two new estimators of the ELBO for mixtures in BBVI, enabling a tremendous reduction in inference time with marginal or even improved impact on performance. Collectively, our contributions enable scalability to hundreds of mixture components and provide superior estimation performance in shorter time, with fewer network parameters compared to previous Mixture VAEs. Experimenting with MISVAE, we achieve astonishing, SOTA results on MNIST. Furthermore, we empirically validate our estimators in other BBVI settings, including Bayesian phylogenetic inference, where we improve inference times for the SOTA mixture model on eight data sets.","url_abs":"https://arxiv.org/abs/2406.07083v1","url_pdf":"https://arxiv.org/pdf/2406.07083v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-mixture-learning-in-black-box","repo_url":"https://github.com/okviman/efficient-mixtures","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"density-estimation","task_name":"Density Estimation"},{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[{"method_slug":"variational-inference","method_name":"Variational Inference"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.07083","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.07083"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/okviman/efficient-mixtures","reach":null}],"summary":{"ran":8,"ran_honours":2,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":12,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":12,"samples":[{"code_sha256_prefix":"ed3b70ab7a6d4b5d","entry":"Conv2d","repo":"okviman/efficient-mixtures","repo_kind":"official","path":"models/misvae.py","file_url":"https://github.com/okviman/efficient-mixtures/blob/HEAD/models/misvae.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ed3b70ab7a6d4b5d"}},{"code_sha256_prefix":"d82fe55119b35e73","entry":"EnsembleGatedConv2dEncoders","repo":"okviman/efficient-mixtures","repo_kind":"official","path":"models/misvae.py","file_url":"https://github.com/okviman/efficient-mixtures/blob/HEAD/models/misvae.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d82fe55119b35e73"}},{"code_sha256_prefix":"7a3bfb98894b2c9c","entry":"GatedConv2d","repo":"okviman/efficient-mixtures","repo_kind":"official","path":"models/misvae.py","file_url":"https://github.com/okviman/efficient-mixtures/blob/HEAD/models/misvae.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7a3bfb98894b2c9c"}},{"code_sha256_prefix":"ff0ad3c747498f46","entry":"GatedConv2dEncoder","repo":"okviman/efficient-mixtures","repo_kind":"official","path":"models/misvae.py","file_url":"https://github.com/okviman/efficient-mixtures/blob/HEAD/models/misvae.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ff0ad3c747498f46"}},{"code_sha256_prefix":"79f679bd8a2777cc","entry":"GatedConv2dResidualEncoder","repo":"okviman/efficient-mixtures","repo_kind":"official","path":"models/misvae.py","file_url":"https://github.com/okviman/efficient-mixtures/blob/HEAD/models/misvae.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"79f679bd8a2777cc"}},{"code_sha256_prefix":"1c2db98cc4262a60","entry":"GatedDense","repo":"okviman/efficient-mixtures","repo_kind":"official","path":"models/misvae.py","file_url":"https://github.com/okviman/efficient-mixtures/blob/HEAD/models/misvae.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1c2db98cc4262a60"}},{"code_sha256_prefix":"1e7fccef55322b25","entry":"MaskedConv2d","repo":"okviman/efficient-mixtures","repo_kind":"official","path":"models/misvae.py","file_url":"https://github.com/okviman/efficient-mixtures/blob/HEAD/models/misvae.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1e7fccef55322b25"}},{"code_sha256_prefix":"1374c4bce3e76e05","entry":"SingleLayerPixelCNNDecoder","repo":"okviman/efficient-mixtures","repo_kind":"official","path":"models/misvae.py","file_url":"https://github.com/okviman/efficient-mixtures/blob/HEAD/models/misvae.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1374c4bce3e76e05"}},{"code_sha256_prefix":"15cef4b080fc9a0d","entry":"evaluate","repo":"okviman/efficient-mixtures","repo_kind":"official","path":"mnist_train.py","file_url":"https://github.com/okviman/efficient-mixtures/blob/HEAD/mnist_train.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"15cef4b080fc9a0d"}},{"code_sha256_prefix":"06d3d6f24a5a0eb4","entry":"evaluate_in_parts","repo":"okviman/efficient-mixtures","repo_kind":"official","path":"mnist_train.py","file_url":"https://github.com/okviman/efficient-mixtures/blob/HEAD/mnist_train.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"06d3d6f24a5a0eb4"}},{"code_sha256_prefix":"7cb76609b75bc07d","entry":"trainer","repo":"okviman/efficient-mixtures","repo_kind":"official","path":"mnist_train.py","file_url":"https://github.com/okviman/efficient-mixtures/blob/HEAD/mnist_train.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"none","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7cb76609b75bc07d"}},{"code_sha256_prefix":"0828e4fc807cceca","entry":"MISVAECNN","repo":"okviman/efficient-mixtures","repo_kind":"official","path":"models/misvae.py","file_url":"https://github.com/okviman/efficient-mixtures/blob/HEAD/models/misvae.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0828e4fc807cceca"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}