{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/resnet-with-one-neuron-hidden-layers-is-a","title":"ResNet with one-neuron hidden layers is a Universal Approximator","arxiv_id":"1806.10909","date":"2018-06-28","proceeding":"NeurIPS 2018 12","authors":["Hongzhou Lin","Stefanie Jegelka"],"abstract":"We demonstrate that a very deep ResNet with stacked modules with one neuron\nper hidden layer and ReLU activation functions can uniformly approximate any\nLebesgue integrable function in $d$ dimensions, i.e. $\\ell_1(\\mathbb{R}^d)$.\nBecause of the identity mapping inherent to ResNets, our network has\nalternating layers of dimension one and $d$. This stands in sharp contrast to\nfully connected networks, which are not universal approximators if their width\nis the input dimension $d$ [Lu et al, 2017; Hanin and Sellke, 2017]. Hence, our\nresult implies an increase in representational power for narrow deep networks\nby the ResNet architecture.","url_abs":"http://arxiv.org/abs/1806.10909v2","url_pdf":"http://arxiv.org/pdf/1806.10909v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"resnet-with-one-neuron-hidden-layers-is-a","repo_url":"https://github.com/sivakon/resnet-approximator","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1806.10909","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}