{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-generative-stochastic-networks-trainable","title":"Deep Generative Stochastic Networks Trainable by Backprop","arxiv_id":"1306.1091","date":"2013-06-05","proceeding":null,"authors":["Yoshua Bengio","Éric Thibodeau-Laufer","Guillaume Alain","Jason Yosinski"],"abstract":"We introduce a novel training principle for probabilistic models that is an\nalternative to maximum likelihood. The proposed Generative Stochastic Networks\n(GSN) framework is based on learning the transition operator of a Markov chain\nwhose stationary distribution estimates the data distribution. The transition\ndistribution of the Markov chain is conditional on the previous state,\ngenerally involving a small move, so this conditional distribution has fewer\ndominant modes, being unimodal in the limit of small moves. Thus, it is easier\nto learn because it is easier to approximate its partition function, more like\nlearning to perform supervised function approximation, with gradients that can\nbe obtained by backprop. We provide theorems that generalize recent work on the\nprobabilistic interpretation of denoising autoencoders and obtain along the way\nan interesting justification for dependency networks and generalized\npseudolikelihood, along with a definition of an appropriate joint distribution\nand sampling mechanism even when the conditionals are not consistent. GSNs can\nbe used with missing inputs and can be used to sample subsets of variables\ngiven the rest. We validate these theoretical results with experiments on two\nimage datasets using an architecture that mimics the Deep Boltzmann Machine\nGibbs sampler but allows training to proceed with simple backprop, without the\nneed for layerwise pretraining.","url_abs":"http://arxiv.org/abs/1306.1091v5","url_pdf":"http://arxiv.org/pdf/1306.1091v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-generative-stochastic-networks-trainable","repo_url":"https://github.com/HUJI-Deep/GSN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"deep-generative-stochastic-networks-trainable","repo_url":"https://github.com/cycentum/bert-based-text-generation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"deep-generative-stochastic-networks-trainable","repo_url":"https://github.com/yaoli/GSN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1306.1091","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1306.1091"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yaoli/GSN","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cycentum/bert-based-text-generation","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/HUJI-Deep/GSN","reach":{"status":"ok"}}],"summary":{"unverified":2},"by_repo_kind":{"listed":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2cb874d469cd529c","entry":"convert_examples_to_features","repo":"cycentum/bert-based-text-generation","repo_kind":"listed","path":"word_replacement/word_replacement.py","file_url":"https://github.com/cycentum/bert-based-text-generation/blob/HEAD/word_replacement/word_replacement.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2cb874d469cd529c"}},{"code_sha256_prefix":"86dfb98ea090a3bd","entry":"input_fn_builder","repo":"cycentum/bert-based-text-generation","repo_kind":"listed","path":"word_replacement/word_replacement.py","file_url":"https://github.com/cycentum/bert-based-text-generation/blob/HEAD/word_replacement/word_replacement.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"86dfb98ea090a3bd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}