{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/vaebm-a-symbiosis-between-variational","title":"VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models","arxiv_id":"2010.00654","date":"2020-10-01","proceeding":"ICLR 2021 1","authors":["Zhisheng Xiao","Karsten Kreis","Jan Kautz","Arash Vahdat"],"abstract":"Energy-based models (EBMs) have recently been successful in representing complex distributions of small images. However, sampling from them requires expensive Markov chain Monte Carlo (MCMC) iterations that mix slowly in high dimensional pixel space. Unlike EBMs, variational autoencoders (VAEs) generate samples quickly and are equipped with a latent space that enables fast traversal of the data manifold. However, VAEs tend to assign high probability density to regions in data space outside the actual data distribution and often fail at generating sharp images. In this paper, we propose VAEBM, a symbiotic composition of a VAE and an EBM that offers the best of both worlds. VAEBM captures the overall mode structure of the data distribution using a state-of-the-art VAE and it relies on its EBM component to explicitly exclude non-data-like regions from the model and refine the image samples. Moreover, the VAE component in VAEBM allows us to speed up MCMC updates by reparameterizing them in the VAE's latent space. Our experimental results show that VAEBM outperforms state-of-the-art VAEs and EBMs in generative quality on several benchmark image datasets by a large margin. It can generate high-quality images as large as 256$\\times$256 pixels with short MCMC chains. We also demonstrate that VAEBM provides complete mode coverage and performs well in out-of-distribution detection. The source code is available at https://github.com/NVlabs/VAEBM","url_abs":"https://arxiv.org/abs/2010.00654v3","url_pdf":"https://arxiv.org/pdf/2010.00654v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"vaebm-a-symbiosis-between-variational","repo_url":"https://github.com/NVlabs/VAEBM","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"out-of-distribution-detection","task_name":"Out-of-Distribution Detection"}],"methods":[{"method_slug":"ebm","method_name":"EBM"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-generation-on-celeba-hq-256x256","task":"Image Generation","dataset":"CelebA-HQ 256x256","model":"VAEBM","rank_in_archive_order":16,"of":19,"metrics":{"FID":"20.38"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-celeba-hq-64x64","task":"Image Generation","dataset":"CelebA-HQ 64x64","model":"VAEBM","rank_in_archive_order":2,"of":3,"metrics":{"FID":"5.31"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-stacked-mnist","task":"Image Generation","dataset":"Stacked MNIST","model":"VAEBM","rank_in_archive_order":1,"of":3,"metrics":{"FID":"12.96","Inception score":"8.15"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2010.00654","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2010.00654"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/NVlabs/VAEBM","reach":null}],"summary":{"ran":2,"ran_honours":1,"ran_draft_wrong":1,"ran_fixture":2,"unverified":3},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"7278a7b671e05581","entry":"Conv2D","repo":"NVlabs/VAEBM","repo_kind":"official","path":"ebm_models.py","file_url":"https://github.com/NVlabs/VAEBM/blob/HEAD/ebm_models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"7278a7b671e05581"}},{"code_sha256_prefix":"8f49ee66eb82532f","entry":"Lip_swish","repo":"NVlabs/VAEBM","repo_kind":"official","path":"ebm_models.py","file_url":"https://github.com/NVlabs/VAEBM/blob/HEAD/ebm_models.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"8f49ee66eb82532f"}},{"code_sha256_prefix":"f9563e26a42cd214","entry":"ResBlock","repo":"NVlabs/VAEBM","repo_kind":"official","path":"ebm_models.py","file_url":"https://github.com/NVlabs/VAEBM/blob/HEAD/ebm_models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f9563e26a42cd214"}},{"code_sha256_prefix":"e68ef3509ca0d2e9","entry":"norm","repo":"NVlabs/VAEBM","repo_kind":"official","path":"ebm_models.py","file_url":"https://github.com/NVlabs/VAEBM/blob/HEAD/ebm_models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"e68ef3509ca0d2e9"}},{"code_sha256_prefix":"509ea43acade44e9","entry":"normalize_weight_jit","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"509ea43acade44e9"}},{"code_sha256_prefix":"f3e2ca7ba6ae5e28","entry":"sample_buffer","repo":"NVlabs/VAEBM","repo_kind":"official","path":"train_VAEBM.py","file_url":"https://github.com/NVlabs/VAEBM/blob/HEAD/train_VAEBM.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f3e2ca7ba6ae5e28"}},{"code_sha256_prefix":"66595f170e26d4b9","entry":"EBM_CelebA64","repo":"NVlabs/VAEBM","repo_kind":"official","path":"ebm_models.py","file_url":"https://github.com/NVlabs/VAEBM/blob/HEAD/ebm_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"66595f170e26d4b9"}},{"code_sha256_prefix":"dd83a8a0b04c47fd","entry":"average_tensor","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"dd83a8a0b04c47fd"}},{"code_sha256_prefix":"18cf305569971c49","entry":"sample_from_EBM","repo":"NVlabs/VAEBM","repo_kind":"official","path":"sample_VAEBM.py","file_url":"https://github.com/NVlabs/VAEBM/blob/HEAD/sample_VAEBM.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"18cf305569971c49"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}