{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/black-box-variational-inference-with-a","title":"Black Box Variational Inference with a Deterministic Objective: Faster, More Accurate, and Even More Black Box","arxiv_id":"2304.05527","date":"2023-04-11","proceeding":null,"authors":["Ryan Giordano","Martin Ingram","Tamara Broderick"],"abstract":"Automatic differentiation variational inference (ADVI) offers fast and easy-to-use posterior approximation in multiple modern probabilistic programming languages. However, its stochastic optimizer lacks clear convergence criteria and requires tuning parameters. Moreover, ADVI inherits the poor posterior uncertainty estimates of mean-field variational Bayes (MFVB). We introduce \"deterministic ADVI\" (DADVI) to address these issues. DADVI replaces the intractable MFVB objective with a fixed Monte Carlo approximation, a technique known in the stochastic optimization literature as the \"sample average approximation\" (SAA). By optimizing an approximate but deterministic objective, DADVI can use off-the-shelf second-order optimization, and, unlike standard mean-field ADVI, is amenable to more accurate posterior covariances via linear response (LR). In contrast to existing worst-case theory, we show that, on certain classes of common statistical problems, DADVI and the SAA can perform well with relatively few samples even in very high dimensions, though we also show that such favorable results cannot extend to variational approximations that are too expressive relative to mean-field ADVI. We show on a variety of real-world problems that DADVI reliably finds good solutions with default settings (unlike ADVI) and, together with LR covariances, is typically faster and more accurate than standard ADVI.","url_abs":"https://arxiv.org/abs/2304.05527v4","url_pdf":"https://arxiv.org/pdf/2304.05527v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"black-box-variational-inference-with-a","repo_url":"https://github.com/rgiordan/dadvipaper","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"probabilistic-programming","task_name":"Probabilistic Programming"},{"task_slug":"stochastic-optimization","task_name":"Stochastic Optimization"},{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[{"method_slug":"variational-inference","method_name":"Variational Inference"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2304.05527","atlas_url":"https://app.syntology.ai/?focus=2304.05527","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2304.05527"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rgiordan/dadvipaper","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8e2ca24e81e52d9c","entry":"add_columns","repo":"rgiordan/dadvipaper","repo_kind":"official","path":"dadvi-experiments/comparison/coverage_helpers.py","file_url":"https://github.com/rgiordan/dadvipaper/blob/HEAD/dadvi-experiments/comparison/coverage_helpers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8e2ca24e81e52d9c"}},{"code_sha256_prefix":"b07775b55f7997c3","entry":"compute_distribution_from_draws","repo":"rgiordan/dadvipaper","repo_kind":"official","path":"dadvi-experiments/comparison/compute_occu_predictions.py","file_url":"https://github.com/rgiordan/dadvipaper/blob/HEAD/dadvi-experiments/comparison/compute_occu_predictions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b07775b55f7997c3"}},{"code_sha256_prefix":"88644168a4c0a0c3","entry":"compute_distribution_from_draws","repo":"rgiordan/dadvipaper","repo_kind":"official","path":"dadvi-experiments/comparison/compute_tennis_matchups.py","file_url":"https://github.com/rgiordan/dadvipaper/blob/HEAD/dadvi-experiments/comparison/compute_tennis_matchups.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"88644168a4c0a0c3"}},{"code_sha256_prefix":"66c3f959eedd095c","entry":"skill_difference","repo":"rgiordan/dadvipaper","repo_kind":"official","path":"dadvi-experiments/comparison/compute_tennis_matchups.py","file_url":"https://github.com/rgiordan/dadvipaper/blob/HEAD/dadvi-experiments/comparison/compute_tennis_matchups.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"66c3f959eedd095c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}