{"url":"/method/sdae","slug":"sdae","name":"SDAE","full_name":"Stacked Denoising Autoencoder","full_name_withheld":false,"description_markdown":"The Stacked Denoising Autoencoder (SdA) is an extension of the stacked autoencoder [Bengio07] and it was introduced in [Vincent08].\r\n\r\nDenoising autoencoders can be stacked to form a deep network by feeding the latent representation (output code) of the [denoising autoencoder](https://paperswithcode.com/method/denoising-autoencoder) found on the layer below as input to the current layer. The unsupervised pre-training of such an architecture is done one layer at a time. Each layer is trained as a denoising autoencoder by minimizing the error in reconstructing its input (which is the output code of the previous layer). Once the first k layers are trained, we can train the k+1-th layer because we can now compute the code or latent representation from the layer below.\r\n\r\nOnce all layers are pre-trained, the network goes through a second stage of training called fine-tuning. Here we consider supervised fine-tuning where we want to minimize prediction error on a supervised task. For this, we first add a [logistic regression](https://paperswithcode.com/method/logistic-regression) layer on top of the network (more precisely on the output code of the output layer). We then train the entire network as we would train a multilayer perceptron. At this point, we only consider the encoding parts of each auto-encoder. This stage is supervised, since now we use the target class during training. (See the Multilayer Perceptron for details on the multilayer perceptron.)\r\n\r\nThis can be easily implemented in Theano, using the class defined previously for a denoising autoencoder. We can see the stacked denoising autoencoder as having two facades: a list of autoencoders, and an MLP. During pre-training we use the first facade, i.e., we treat our model as a list of autoencoders, and train each autoencoder seperately. In the second stage of training, we use the second facade. These two facades are linked because:\r\n* the autoencoders and the sigmoid layers of the MLP share parameters, and\r\n* the latent representations computed by intermediate layers of the MLP are fed as input to the autoencoders.\r\n\r\nExtracted from [webpage](http://deeplearning.net/tutorial/SdA.html)\r\n\r\nImage: [Jigar Bandaria](https://miro.medium.com/max/701/1*wbaL5CvUkVkZxlSUsRS5IQ.png)\r\n\r\n**Source**:\r\n\r\nImage: [Jigar Bandaria](https://blog.insightdatascience.com/brain-mri-image-segmentation-using-stacked-denoising-autoencoders-4e91417688f6)\r\n\r\nWebpage: [deeplearning.net](http://deeplearning.net/tutorial/SdA.html)\r\n\r\nWebpage: [www.iro.umontreal.ca](http://www.iro.umontreal.ca/~pift6266/H10/notes/SdA.html)\r\n\r\nPaper:\r\n\r\n[Vincent, H. Larochelle Y. Bengio and P.A. Manzagol, Extracting and Composing Robust Features with Denoising Autoencoders](https://doi.org/10.1145/1390156.1390294)\r\n\r\n[Vincent, H. Larochelle Y. Bengio and P.A. Manzagol, Extracting and Composing Robust Features with Denoising Autoencoders](http://www.iro.umontreal.ca/~lisa/publications2/index.php/publications/show/217)","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":null,"title":null,"url_on_a_paper_host":false},"code_snippet_url":"https://github.com/lisa-lab/DeepLearningTutorials/blob/master/code/SdA.py","code_snippet_url_on_a_code_host":true,"categories":[{"area":"Computer Vision","area_id":"computer-vision","collection":"Generative Models","url":"/methods/category/generative-models","pwc_aliases":[]}],"n_papers_tagged":5,"archive_num_papers":null,"papers_newest_first":[{"paper":"/paper/sdae-self-distillated-masked-autoencoder","title":"SdAE: Self-distillated Masked Autoencoder","date":"2022-07-31","arxiv_id":"2208.00449","n_code_links":1,"syntology":{"ran":0,"of":1,"unverified":1,"pointer_only":0}},{"paper":null,"title":"Fast Calculation of Probabilistic Optimal Power Flow: A Deep Learning Approach","date":"2019-06-24","arxiv_id":"1906.09951","n_code_links":0,"syntology":null},{"paper":null,"title":"Encoding Multi-Resolution Brain Networks Using Unsupervised Deep Learning","date":"2017-08-13","arxiv_id":"1708.04232","n_code_links":0,"syntology":null},{"paper":null,"title":"Semi-supervised Learning using Denoising Autoencoders for Brain Lesion Detection and Segmentation","date":"2016-11-26","arxiv_id":"1611.08664","n_code_links":0,"syntology":null},{"paper":null,"title":"Real-time Dynamic MRI Reconstruction using Stacked Denoising Autoencoder","date":"2015-03-22","arxiv_id":"1503.06383","n_code_links":0,"syntology":null}],"papers_shown":5,"tasks":[{"task":"/task/denoising","name":"Denoising","papers":3},{"task":"/task/deep-learning","name":"Deep Learning","papers":1},{"task":"/task/descriptive","name":"Descriptive","papers":1},{"task":"/task/lesion-detection","name":"Lesion Detection","papers":1},{"task":"/task/mri-reconstruction","name":"MRI Reconstruction","papers":1},{"task":"/task/segmentation","name":"Segmentation","papers":1},{"task":"/task/self-supervised-learning","name":"Self-Supervised Learning","papers":1},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":1},{"task":"/task/compressed-sensing","name":"compressed sensing","papers":1}],"tasks_shown":9,"n_tasks":9,"usage_by_year":[{"year":"2015","papers":1},{"year":"2016","papers":1},{"year":"2017","papers":1},{"year":"2019","papers":1},{"year":"2022","papers":1}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/sdae"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}