{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/end-to-end-training-of-deep-boltzmann","title":"End-to-end Training of Deep Boltzmann Machines by Unbiased Contrastive Divergence with Local Mode Initialization","arxiv_id":"2305.19684","date":"2023-05-31","proceeding":null,"authors":["Shohei Taniguchi","Masahiro Suzuki","Yusuke Iwasawa","Yutaka Matsuo"],"abstract":"We address the problem of biased gradient estimation in deep Boltzmann machines (DBMs). The existing method to obtain an unbiased estimator uses a maximal coupling based on a Gibbs sampler, but when the state is high-dimensional, it takes a long time to converge. In this study, we propose to use a coupling based on the Metropolis-Hastings (MH) and to initialize the state around a local mode of the target distribution. Because of the propensity of MH to reject proposals, the coupling tends to converge in only one step with a high probability, leading to high efficiency. We find that our method allows DBMs to be trained in an end-to-end fashion without greedy pretraining. We also propose some practical techniques to further improve the performance of DBMs. We empirically demonstrate that our training algorithm enables DBMs to show comparable generative performance to other deep generative models, achieving the FID score of 10.33 for MNIST.","url_abs":"https://arxiv.org/abs/2305.19684v1","url_pdf":"https://arxiv.org/pdf/2305.19684v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"end-to-end-training-of-deep-boltzmann","repo_url":"https://github.com/ishohei220/unbiased_dbm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.19684","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.19684"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ishohei220/unbiased_dbm","reach":null}],"summary":{"ran":1,"ran_draft_wrong":4},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"376afebcff95d9c0","entry":"DBM","repo":"ishohei220/unbiased_dbm","repo_kind":"official","path":"model.py","file_url":"https://github.com/ishohei220/unbiased_dbm/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"376afebcff95d9c0"}},{"code_sha256_prefix":"a84eab69e76bed6d","entry":"bit2float","repo":"ishohei220/unbiased_dbm","repo_kind":"official","path":"model.py","file_url":"https://github.com/ishohei220/unbiased_dbm/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a84eab69e76bed6d"}},{"code_sha256_prefix":"40a50617f7cc0940","entry":"bit2int","repo":"ishohei220/unbiased_dbm","repo_kind":"official","path":"model.py","file_url":"https://github.com/ishohei220/unbiased_dbm/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"40a50617f7cc0940"}},{"code_sha256_prefix":"467eb8a09ce9750f","entry":"float2bit","repo":"ishohei220/unbiased_dbm","repo_kind":"official","path":"model.py","file_url":"https://github.com/ishohei220/unbiased_dbm/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"467eb8a09ce9750f"}},{"code_sha256_prefix":"1f6a75ed80783096","entry":"int2bit","repo":"ishohei220/unbiased_dbm","repo_kind":"official","path":"model.py","file_url":"https://github.com/ishohei220/unbiased_dbm/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1f6a75ed80783096"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}