{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sharing-residual-units-through-collective","title":"Sharing Residual Units Through Collective Tensor Factorization in Deep Neural Networks","arxiv_id":"1703.02180","date":"2017-03-07","proceeding":null,"authors":["Chen Yunpeng","Jin Xiaojie","Kang Bingyi","Feng Jiashi","Yan Shuicheng"],"abstract":"Residual units are wildly used for alleviating optimization difficulties when\nbuilding deep neural networks. However, the performance gain does not well\ncompensate the model size increase, indicating low parameter efficiency in\nthese residual units. In this work, we first revisit the residual function in\nseveral variations of residual units and demonstrate that these residual\nfunctions can actually be explained with a unified framework based on\ngeneralized block term decomposition. Then, based on the new explanation, we\npropose a new architecture, Collective Residual Unit (CRU), which enhances the\nparameter efficiency of deep neural networks through collective tensor\nfactorization. CRU enables knowledge sharing across different residual units\nusing shared factors. Experimental results show that our proposed CRU Network\ndemonstrates outstanding parameter efficiency, achieving comparable\nclassification performance to ResNet-200 with the model size of ResNet-50. By\nbuilding a deeper network using CRU, we can achieve state-of-the-art single\nmodel classification accuracy on ImageNet-1k and Places365-Standard benchmark\ndatasets. (Code and trained models are available on GitHub)","url_abs":"http://arxiv.org/abs/1703.02180v2","url_pdf":"http://arxiv.org/pdf/1703.02180v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sharing-residual-units-through-collective","repo_url":"https://github.com/cypw/CRU-Net","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null},{"paper_slug":"sharing-residual-units-through-collective","repo_url":"https://github.com/osmr/imgclsmob","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mxnet","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"classification","task_name":"General Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}