{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generalized-denoising-auto-encoders-as","title":"Generalized Denoising Auto-Encoders as Generative Models","arxiv_id":"1305.6663","date":"2013-05-29","proceeding":"NeurIPS 2013 12","authors":["Yoshua Bengio","Li Yao","Guillaume Alain","Pascal Vincent"],"abstract":"Recent work has shown how denoising and contractive autoencoders implicitly\ncapture the structure of the data-generating density, in the case where the\ncorruption noise is Gaussian, the reconstruction error is the squared error,\nand the data is continuous-valued. This has led to various proposals for\nsampling from this implicitly learned density function, using Langevin and\nMetropolis-Hastings MCMC. However, it remained unclear how to connect the\ntraining procedure of regularized auto-encoders to the implicit estimation of\nthe underlying data-generating distribution when the data are discrete, or\nusing other forms of corruption process and reconstruction errors. Another\nissue is the mathematical justification which is only valid in the limit of\nsmall corruption noise. We propose here a different attack on the problem,\nwhich deals with all these issues: arbitrary (but noisy enough) corruption,\narbitrary reconstruction loss (seen as a log-likelihood), handling both\ndiscrete and continuous-valued variables, and removing the bias due to\nnon-infinitesimal corruption noise (or non-infinitesimal contractive penalty).","url_abs":"http://arxiv.org/abs/1305.6663v4","url_pdf":"http://arxiv.org/pdf/1305.6663v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generalized-denoising-auto-encoders-as","repo_url":"https://github.com/cycentum/bert-based-text-generation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":null,"task_name":"valid"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1305.6663","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}