{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/beyond-elbos-a-large-scale-evaluation-of","title":"Beyond ELBOs: A Large-Scale Evaluation of Variational Methods for Sampling","arxiv_id":"2406.07423","date":"2024-06-11","proceeding":null,"authors":["Denis Blessing","Xiaogang Jia","Johannes Esslinger","Francisco Vargas","Gerhard Neumann"],"abstract":"Monte Carlo methods, Variational Inference, and their combinations play a pivotal role in sampling from intractable probability distributions. However, current studies lack a unified evaluation framework, relying on disparate performance measures and limited method comparisons across diverse tasks, complicating the assessment of progress and hindering the decision-making of practitioners. In response to these challenges, our work introduces a benchmark that evaluates sampling methods using a standardized task suite and a broad range of performance criteria. Moreover, we study existing metrics for quantifying mode collapse and introduce novel metrics for this purpose. Our findings provide insights into strengths and weaknesses of existing sampling methods, serving as a valuable reference for future developments. The code is publicly available here.","url_abs":"https://arxiv.org/abs/2406.07423v1","url_pdf":"https://arxiv.org/pdf/2406.07423v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"beyond-elbos-a-large-scale-evaluation-of","repo_url":"https://github.com/denisbless/variational_sampling_methods","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[{"method_slug":"variational-inference","method_name":"Variational Inference"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2406.07423","atlas_url":"https://app.syntology.ai/?focus=2406.07423","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.07423"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/denisbless/variational_sampling_methods","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":9,"unverified":1},"by_repo_kind":{"official":{"samples":10,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2e95829a31e8dc4b","entry":"compute_reverse_ess","repo":"denisbless/variational_sampling_methods","repo_kind":"official","path":"algorithms/common/eval_methods/utils.py","file_url":"https://github.com/denisbless/variational_sampling_methods/blob/HEAD/algorithms/common/eval_methods/utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2e95829a31e8dc4b"}},{"code_sha256_prefix":"e5de8a47e5b6f3bb","entry":"create_point","repo":"denisbless/variational_sampling_methods","repo_kind":"official","path":"algorithms/fab/sampling/base.py","file_url":"https://github.com/denisbless/variational_sampling_methods/blob/HEAD/algorithms/fab/sampling/base.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e5de8a47e5b6f3bb"}},{"code_sha256_prefix":"4a97143dac6c9fc8","entry":"extract_last_entry","repo":"denisbless/variational_sampling_methods","repo_kind":"official","path":"algorithms/common/eval_methods/utils.py","file_url":"https://github.com/denisbless/variational_sampling_methods/blob/HEAD/algorithms/common/eval_methods/utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4a97143dac6c9fc8"}},{"code_sha256_prefix":"34677d7bdda50827","entry":"get_grad_intermediate_log_prob","repo":"denisbless/variational_sampling_methods","repo_kind":"official","path":"algorithms/fab/sampling/base.py","file_url":"https://github.com/denisbless/variational_sampling_methods/blob/HEAD/algorithms/fab/sampling/base.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"34677d7bdda50827"}},{"code_sha256_prefix":"de01b4b20037266a","entry":"get_intermediate_log_prob","repo":"denisbless/variational_sampling_methods","repo_kind":"official","path":"algorithms/fab/sampling/base.py","file_url":"https://github.com/denisbless/variational_sampling_methods/blob/HEAD/algorithms/fab/sampling/base.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de01b4b20037266a"}},{"code_sha256_prefix":"2a30e0be0e543a3b","entry":"moving_averages","repo":"denisbless/variational_sampling_methods","repo_kind":"official","path":"algorithms/common/eval_methods/utils.py","file_url":"https://github.com/denisbless/variational_sampling_methods/blob/HEAD/algorithms/common/eval_methods/utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2a30e0be0e543a3b"}},{"code_sha256_prefix":"88fe97b3c30bcc9b","entry":"rejection_sampling","repo":"denisbless/variational_sampling_methods","repo_kind":"official","path":"algorithms/fab/sampling/rejection_sampling.py","file_url":"https://github.com/denisbless/variational_sampling_methods/blob/HEAD/algorithms/fab/sampling/rejection_sampling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"88fe97b3c30bcc9b"}},{"code_sha256_prefix":"4a1953fc2385ba92","entry":"simple_resampling","repo":"denisbless/variational_sampling_methods","repo_kind":"official","path":"algorithms/fab/sampling/resampling.py","file_url":"https://github.com/denisbless/variational_sampling_methods/blob/HEAD/algorithms/fab/sampling/resampling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4a1953fc2385ba92"}},{"code_sha256_prefix":"b2c1edb04bec461b","entry":"smc_simple_resampling","repo":"denisbless/variational_sampling_methods","repo_kind":"official","path":"algorithms/fab/sampling/resampling.py","file_url":"https://github.com/denisbless/variational_sampling_methods/blob/HEAD/algorithms/fab/sampling/resampling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b2c1edb04bec461b"}},{"code_sha256_prefix":"821d3fb722c0d48d","entry":"log_effective_sample_size","repo":"denisbless/variational_sampling_methods","repo_kind":"official","path":"algorithms/fab/sampling/resampling.py","file_url":"https://github.com/denisbless/variational_sampling_methods/blob/HEAD/algorithms/fab/sampling/resampling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"821d3fb722c0d48d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}