{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-implicitly-recurrent-cnns-through","title":"Learning Implicitly Recurrent CNNs Through Parameter Sharing","arxiv_id":"1902.09701","date":"2019-02-26","proceeding":"ICLR 2019 5","authors":["Pedro Savarese","Michael Maire"],"abstract":"We introduce a parameter sharing scheme, in which different layers of a\nconvolutional neural network (CNN) are defined by a learned linear combination\nof parameter tensors from a global bank of templates. Restricting the number of\ntemplates yields a flexible hybridization of traditional CNNs and recurrent\nnetworks. Compared to traditional CNNs, we demonstrate substantial parameter\nsavings on standard image classification tasks, while maintaining accuracy.\n  Our simple parameter sharing scheme, though defined via soft weights, in\npractice often yields trained networks with near strict recurrent structure;\nwith negligible side effects, they convert into networks with actual loops.\nTraining these networks thus implicitly involves discovery of suitable\nrecurrent architectures. Though considering only the design aspect of recurrent\nlinks, our trained networks achieve accuracy competitive with those built using\nstate-of-the-art neural architecture search (NAS) procedures.\n  Our hybridization of recurrent and convolutional networks may also represent\na beneficial architectural bias. Specifically, on synthetic tasks which are\nalgorithmic in nature, our hybrid networks both train faster and extrapolate\nbetter to test examples outside the span of the training set.","url_abs":"http://arxiv.org/abs/1902.09701v2","url_pdf":"http://arxiv.org/pdf/1902.09701v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-implicitly-recurrent-cnns-through","repo_url":"https://github.com/lolemacs/soft-sharing","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-cifar-10","task":"Image Classification","dataset":"CIFAR-10","model":"Shared WRN","rank_in_archive_order":80,"of":265,"metrics":{"Percentage correct":"97.47"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-cifar-100","task":"Image Classification","dataset":"CIFAR-100","model":"Shared WRN","rank_in_archive_order":102,"of":211,"metrics":{"Percentage correct":"82.57"},"uses_additional_data":true},{"leaderboard":"/sota/neural-architecture-search-on-cifar-10","task":"Neural Architecture Search","dataset":"CIFAR-10","model":"Soft Parameter Sharing","rank_in_archive_order":22,"of":41,"metrics":{"Search Time (GPU days)":"0.7","Top-1 Error Rate":"2.53%"},"uses_additional_data":false},{"leaderboard":"/sota/architecture-search-on-cifar-10-image","task":"Neural Architecture Search","dataset":"CIFAR-10 Image Classification","model":"Soft Parameter Sharing","rank_in_archive_order":12,"of":19,"metrics":{"Params":"33.5","Percentage error":"2.53"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1902.09701","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}