{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dense-xunit-networks","title":"Dense xUnit Networks","arxiv_id":"1811.11051","date":"2018-11-27","proceeding":null,"authors":["Idan Kligvasser","Tomer Michaeli"],"abstract":"Deep net architectures have constantly evolved over the past few years,\nleading to significant advancements in a wide array of computer vision tasks.\nHowever, besides high accuracy, many applications also require a low\ncomputational load and limited memory footprint. To date, efficiency has\ntypically been achieved either by architectural choices at the macro level\n(e.g. using skip connections or pruning techniques) or modifications at the\nlevel of the individual layers (e.g. using depth-wise convolutions or channel\nshuffle operations). Interestingly, much less attention has been devoted to the\nrole of the activation functions in constructing efficient nets. Recently,\nKligvasser et al. showed that incorporating spatial connections within the\nactivation functions, enables a significant boost in performance in image\nrestoration tasks, at any given budget of parameters. However, the\neffectiveness of their xUnit module has only been tested on simple small\nmodels, which are not characteristic of those used in high-level vision tasks.\nIn this paper, we adopt and improve the xUnit activation, show how it can be\nincorporated into the DenseNet architecture, and illustrate its high\neffectiveness for classification and image restoration tasks alike. While the\nDenseNet architecture is extremely efficient to begin with, our dense xUnit net\n(DxNet) can typically achieve the same performance with far fewer parameters.\nFor example, on ImageNet, our DxNet outperforms a ReLU-based DenseNet having\n30% more parameters and achieves state-of-the-art results for this budget of\nparameters. Furthermore, in denoising and super-resolution, DxNet significantly\nimproves upon all existing lightweight solutions, including the xUnit-based\nnets of Kligvasser et al.","url_abs":"http://arxiv.org/abs/1811.11051v1","url_pdf":"http://arxiv.org/pdf/1811.11051v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dense-xunit-networks","repo_url":"https://github.com/kligvasser/xUnit","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"image-restoration","task_name":"Image Restoration"},{"task_slug":"super-resolution","task_name":"Super-Resolution"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"concatenated-skip-connection","method_name":"Concatenated Skip Connection"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-block","method_name":"Dense Block"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"pruning","method_name":"Pruning"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}