{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-decoder-concise-image-representations","title":"Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks","arxiv_id":"1810.03982","date":"2018-10-02","proceeding":"ICLR 2019 5","authors":["Reinhard Heckel","Paul Hand"],"abstract":"Deep neural networks, in particular convolutional neural networks, have\nbecome highly effective tools for compressing images and solving inverse\nproblems including denoising, inpainting, and reconstruction from few and noisy\nmeasurements. This success can be attributed in part to their ability to\nrepresent and generate natural images well. Contrary to classical tools such as\nwavelets, image-generating deep neural networks have a large number of\nparameters---typically a multiple of their output dimension---and need to be\ntrained on large datasets. In this paper, we propose an untrained simple image\nmodel, called the deep decoder, which is a deep neural network that can\ngenerate natural images from very few weight parameters. The deep decoder has a\nsimple architecture with no convolutions and fewer weight parameters than the\noutput dimensionality. This underparameterization enables the deep decoder to\ncompress images into a concise set of network weights, which we show is on par\nwith wavelet-based thresholding. Further, underparameterization provides a\nbarrier to overfitting, allowing the deep decoder to have state-of-the-art\nperformance for denoising. The deep decoder is simple in the sense that each\nlayer has an identical structure that consists of only one upsampling unit,\npixel-wise linear combination of channels, ReLU activation, and channelwise\nnormalization. This simplicity makes the network amenable to theoretical\nanalysis, and it sheds light on the aspects of neural networks that enable them\nto form effective signal representations.","url_abs":"http://arxiv.org/abs/1810.03982v1","url_pdf":"http://arxiv.org/pdf/1810.03982v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-decoder-concise-image-representations","repo_url":"https://github.com/reinhardh/supplement_deep_decoder","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"deep-decoder-concise-image-representations","repo_url":"https://github.com/GauriJagatap/invimaging-deeppriors","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"deep-decoder-concise-image-representations","repo_url":"https://github.com/Martinchen2015/VideoDDtest","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"deep-decoder-concise-image-representations","repo_url":"https://github.com/TanviKulkarni07/CS-MRI-Recon","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"denoising","task_name":"Denoising"}],"methods":[{"method_slug":"relu","method_name":"ReLU"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1810.03982","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}