{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-training-of-energy-based-models-1","title":"Efficient Training of Energy-Based Models Using Jarzynski Equality","arxiv_id":"2305.19414","date":"2023-05-30","proceeding":"NeurIPS 2023 11","authors":["Davide Carbone","Mengjian Hua","Simon Coste","Eric Vanden-Eijnden"],"abstract":"Energy-based models (EBMs) are generative models inspired by statistical physics with a wide range of applications in unsupervised learning. Their performance is best measured by the cross-entropy (CE) of the model distribution relative to the data distribution. Using the CE as the objective for training is however challenging because the computation of its gradient with respect to the model parameters requires sampling the model distribution. Here we show how results for nonequilibrium thermodynamics based on Jarzynski equality together with tools from sequential Monte-Carlo sampling can be used to perform this computation efficiently and avoid the uncontrolled approximations made using the standard contrastive divergence algorithm. Specifically, we introduce a modification of the unadjusted Langevin algorithm (ULA) in which each walker acquires a weight that enables the estimation of the gradient of the cross-entropy at any step during GD, thereby bypassing sampling biases induced by slow mixing of ULA. We illustrate these results with numerical experiments on Gaussian mixture distributions as well as the MNIST dataset. We show that the proposed approach outperforms methods based on the contrastive divergence algorithm in all the considered situations.","url_abs":"https://arxiv.org/abs/2305.19414v2","url_pdf":"https://arxiv.org/pdf/2305.19414v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-training-of-energy-based-models-1","repo_url":"https://github.com/submissionx12/ebms_jarzynski","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2305.19414","atlas_url":"https://app.syntology.ai/?focus=2305.19414","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.19414"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/submissionx12/ebms_jarzynski","reach":null}],"summary":{"ran_honours":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"6f88350329b0e1de","entry":"U","repo":"submissionx12/ebms_jarzynski","repo_kind":"official","path":"GMM_code/GMM.py","file_url":"https://github.com/submissionx12/ebms_jarzynski/blob/HEAD/GMM_code/GMM.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6f88350329b0e1de"}},{"code_sha256_prefix":"b39343c2321616ea","entry":"dUdx","repo":"submissionx12/ebms_jarzynski","repo_kind":"official","path":"GMM_code/GMM.py","file_url":"https://github.com/submissionx12/ebms_jarzynski/blob/HEAD/GMM_code/GMM.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b39343c2321616ea"}},{"code_sha256_prefix":"23b7e075c202122b","entry":"rho","repo":"submissionx12/ebms_jarzynski","repo_kind":"official","path":"GMM_code/GMM.py","file_url":"https://github.com/submissionx12/ebms_jarzynski/blob/HEAD/GMM_code/GMM.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"23b7e075c202122b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}