{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/repr-improved-training-of-convolutional","title":"RePr: Improved Training of Convolutional Filters","arxiv_id":"1811.07275","date":"2018-11-18","proceeding":"CVPR 2019 6","authors":["Aaditya Prakash","James Storer","Dinei Florencio","Cha Zhang"],"abstract":"A well-trained Convolutional Neural Network can easily be pruned without\nsignificant loss of performance. This is because of unnecessary overlap in the\nfeatures captured by the network's filters. Innovations in network architecture\nsuch as skip/dense connections and Inception units have mitigated this problem\nto some extent, but these improvements come with increased computation and\nmemory requirements at run-time. We attempt to address this problem from\nanother angle - not by changing the network structure but by altering the\ntraining method. We show that by temporarily pruning and then restoring a\nsubset of the model's filters, and repeating this process cyclically, overlap\nin the learned features is reduced, producing improved generalization. We show\nthat the existing model-pruning criteria are not optimal for selecting filters\nto prune in this context and introduce inter-filter orthogonality as the\nranking criteria to determine under-expressive filters. Our method is\napplicable both to vanilla convolutional networks and more complex modern\narchitectures, and improves the performance across a variety of tasks,\nespecially when applied to smaller networks.","url_abs":"http://arxiv.org/abs/1811.07275v3","url_pdf":"http://arxiv.org/pdf/1811.07275v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"repr-improved-training-of-convolutional","repo_url":"https://github.com/siahuat0727/RePr","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1811.07275","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}