{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/energy-based-calibrated-vae-with-test-time","title":"Energy-Calibrated VAE with Test Time Free Lunch","arxiv_id":"2311.04071","date":"2023-11-07","proceeding":null,"authors":["Yihong Luo","Siya Qiu","Xingjian Tao","Yujun Cai","Jing Tang"],"abstract":"In this paper, we propose a novel generative model that utilizes a conditional Energy-Based Model (EBM) for enhancing Variational Autoencoder (VAE), termed Energy-Calibrated VAE (EC-VAE). Specifically, VAEs often suffer from blurry generated samples due to the lack of a tailored training on the samples generated in the generative direction. On the other hand, EBMs can generate high-quality samples but require expensive Markov Chain Monte Carlo (MCMC) sampling. To address these issues, we introduce a conditional EBM for calibrating the generative direction of VAE during training, without requiring it for the generation at test time. In particular, we train EC-VAE upon both the input data and the calibrated samples with adaptive weight to enhance efficacy while avoiding MCMC sampling at test time. Furthermore, we extend the calibration idea of EC-VAE to variational learning and normalizing flows, and apply EC-VAE to an additional application of zero-shot image restoration via neural transport prior and range-null theory. We evaluate the proposed method with two applications, including image generation and zero-shot image restoration, and the experimental results show that our method achieves competitive performance over single-step non-adversarial generation. Our code is available at https://github.com/DJ-LYH/EC-VAE.","url_abs":"https://arxiv.org/abs/2311.04071v5","url_pdf":"https://arxiv.org/pdf/2311.04071v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"energy-based-calibrated-vae-with-test-time","repo_url":"https://github.com/DJ-LYH/EC-VAE","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"image-restoration","task_name":"Image Restoration"}],"methods":[{"method_slug":"ebm","method_name":"EBM"},{"method_slug":"normalizing-flows","method_name":"Normalizing Flows"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2311.04071","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.04071"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/DJ-LYH/EC-VAE","reach":{"status":"ok"}}],"summary":{"ran_honours":2,"unverified":5},"by_repo_kind":{"official":{"samples":7,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"6154e3744ece5728","entry":"create_window","repo":"DJ-LYH/EC-VAE","repo_kind":"official","path":"loss/ssim.py","file_url":"https://github.com/DJ-LYH/EC-VAE/blob/HEAD/loss/ssim.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6154e3744ece5728"}},{"code_sha256_prefix":"c56b7ef16f309a45","entry":"gaussian","repo":"DJ-LYH/EC-VAE","repo_kind":"official","path":"loss/ssim.py","file_url":"https://github.com/DJ-LYH/EC-VAE/blob/HEAD/loss/ssim.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c56b7ef16f309a45"}},{"code_sha256_prefix":"8a32c0b893fc5a42","entry":"distance_metric","repo":"DJ-LYH/EC-VAE","repo_kind":"official","path":"loss/per.py","file_url":"https://github.com/DJ-LYH/EC-VAE/blob/HEAD/loss/per.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8a32c0b893fc5a42"}},{"code_sha256_prefix":"ef65c7ad511bfa66","entry":"normalize_tensor","repo":"DJ-LYH/EC-VAE","repo_kind":"official","path":"loss/deep_loss.py","file_url":"https://github.com/DJ-LYH/EC-VAE/blob/HEAD/loss/deep_loss.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ef65c7ad511bfa66"}},{"code_sha256_prefix":"0086e76caac76ff6","entry":"normalize_tensor","repo":"DJ-LYH/EC-VAE","repo_kind":"official","path":"loss/watson_vgg.py","file_url":"https://github.com/DJ-LYH/EC-VAE/blob/HEAD/loss/watson_vgg.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0086e76caac76ff6"}},{"code_sha256_prefix":"cc287dc402b9202b","entry":"softmax","repo":"DJ-LYH/EC-VAE","repo_kind":"official","path":"loss/watson.py","file_url":"https://github.com/DJ-LYH/EC-VAE/blob/HEAD/loss/watson.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cc287dc402b9202b"}},{"code_sha256_prefix":"9151c793f90473f0","entry":"ssim","repo":"DJ-LYH/EC-VAE","repo_kind":"official","path":"loss/ssim.py","file_url":"https://github.com/DJ-LYH/EC-VAE/blob/HEAD/loss/ssim.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9151c793f90473f0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}