{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/reweighted-wake-sleep","title":"Reweighted Wake-Sleep","arxiv_id":"1406.2751","date":"2014-06-11","proceeding":null,"authors":["Jörg Bornschein","Yoshua Bengio"],"abstract":"Training deep directed graphical models with many hidden variables and\nperforming inference remains a major challenge. Helmholtz machines and deep\nbelief networks are such models, and the wake-sleep algorithm has been proposed\nto train them. The wake-sleep algorithm relies on training not just the\ndirected generative model but also a conditional generative model (the\ninference network) that runs backward from visible to latent, estimating the\nposterior distribution of latent given visible. We propose a novel\ninterpretation of the wake-sleep algorithm which suggests that better\nestimators of the gradient can be obtained by sampling latent variables\nmultiple times from the inference network. This view is based on importance\nsampling as an estimator of the likelihood, with the approximate inference\nnetwork as a proposal distribution. This interpretation is confirmed\nexperimentally, showing that better likelihood can be achieved with this\nreweighted wake-sleep procedure. Based on this interpretation, we propose that\na sigmoidal belief network is not sufficiently powerful for the layers of the\ninference network in order to recover a good estimator of the posterior\ndistribution of latent variables. Our experiments show that using a more\npowerful layer model, such as NADE, yields substantially better generative\nmodels.","url_abs":"http://arxiv.org/abs/1406.2751v4","url_pdf":"http://arxiv.org/pdf/1406.2751v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"reweighted-wake-sleep","repo_url":"https://github.com/jbornschein/reweighted-ws","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"reweighted-wake-sleep","repo_url":"https://github.com/kpoeppel/pytorch_probgraph","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1406.2751","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1406.2751"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kpoeppel/pytorch_probgraph","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jbornschein/reweighted-ws","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1},"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"fc88f487990b9397","entry":"logsumexp","repo":"kpoeppel/pytorch_probgraph","repo_kind":"listed","path":"pytorch_probgraph/hm.py","file_url":"https://github.com/kpoeppel/pytorch_probgraph/blob/HEAD/pytorch_probgraph/hm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"fc88f487990b9397"}},{"code_sha256_prefix":"dc33b402e74e0ff1","entry":"enumerate_pairs","repo":"jbornschein/reweighted-ws","repo_kind":"official","path":"learning/models/rws.py","file_url":"https://github.com/jbornschein/reweighted-ws/blob/HEAD/learning/models/rws.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"dc33b402e74e0ff1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}