{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-higher-ranks-via-adversarial-weight-1","title":"Towards Higher Ranks via Adversarial Weight Pruning","arxiv_id":"2311.17493","date":"2023-11-29","proceeding":"NeurIPS 2023 11","authors":["Yuchuan Tian","Hanting Chen","Tianyu Guo","Chao Xu","Yunhe Wang"],"abstract":"Convolutional Neural Networks (CNNs) are hard to deploy on edge devices due to its high computation and storage complexities. As a common practice for model compression, network pruning consists of two major categories: unstructured and structured pruning, where unstructured pruning constantly performs better. However, unstructured pruning presents a structured pattern at high pruning rates, which limits its performance. To this end, we propose a Rank-based PruninG (RPG) method to maintain the ranks of sparse weights in an adversarial manner. In each step, we minimize the low-rank approximation error for the weight matrices using singular value decomposition, and maximize their distance by pushing the weight matrices away from its low rank approximation. This rank-based optimization objective guides sparse weights towards a high-rank topology. The proposed method is conducted in a gradual pruning fashion to stabilize the change of rank during training. Experimental results on various datasets and different tasks demonstrate the effectiveness of our algorithm in high sparsity. The proposed RPG outperforms the state-of-the-art performance by 1.13% top-1 accuracy on ImageNet in ResNet-50 with 98% sparsity. The codes are available at https://github.com/huawei-noah/Efficient-Computing/tree/master/Pruning/RPG and https://gitee.com/mindspore/models/tree/master/research/cv/RPG.","url_abs":"https://arxiv.org/abs/2311.17493v1","url_pdf":"https://arxiv.org/pdf/2311.17493v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-higher-ranks-via-adversarial-weight-1","repo_url":"https://github.com/huawei-noah/Efficient-Computing","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"model-compression","task_name":"Model Compression"},{"task_slug":"network-pruning","task_name":"Network Pruning"}],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2311.17493","atlas_url":"https://app.syntology.ai/?focus=2311.17493","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.17493"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huawei-noah/Efficient-Computing","reach":null}],"summary":{"ran_fixture":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"7f969fc7f54308c2","entry":"bsearch","repo":"huawei-noah/Efficient-Computing","repo_kind":"official","path":"Pruning/RPG/image_classification/rank_kit.py","file_url":"https://github.com/huawei-noah/Efficient-Computing/blob/HEAD/Pruning/RPG/image_classification/rank_kit.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7f969fc7f54308c2"}},{"code_sha256_prefix":"4612336ecb286fbe","entry":"find_k","repo":"huawei-noah/Efficient-Computing","repo_kind":"official","path":"Pruning/RPG/image_classification/rank_kit.py","file_url":"https://github.com/huawei-noah/Efficient-Computing/blob/HEAD/Pruning/RPG/image_classification/rank_kit.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4612336ecb286fbe"}},{"code_sha256_prefix":"c6a3f0f5e7b54927","entry":"get_sing_loss","repo":"huawei-noah/Efficient-Computing","repo_kind":"official","path":"Pruning/RPG/image_classification/rank_kit.py","file_url":"https://github.com/huawei-noah/Efficient-Computing/blob/HEAD/Pruning/RPG/image_classification/rank_kit.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c6a3f0f5e7b54927"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}