{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/trp-trained-rank-pruning-for-efficient-deep","title":"TRP: Trained Rank Pruning for Efficient Deep Neural Networks","arxiv_id":"2004.14566","date":"2020-04-30","proceeding":null,"authors":["Yuhui Xu","Yuxi Li","Shuai Zhang","Wei Wen","Botao Wang","Yingyong Qi","Yiran Chen","Weiyao Lin","Hongkai Xiong"],"abstract":"To enable DNNs on edge devices like mobile phones, low-rank approximation has been widely adopted because of its solid theoretical rationale and efficient implementations. Several previous works attempted to directly approximate a pretrained model by low-rank decomposition; however, small approximation errors in parameters can ripple over a large prediction loss. As a result, performance usually drops significantly and a sophisticated effort on fine-tuning is required to recover accuracy. Apparently, it is not optimal to separate low-rank approximation from training. Unlike previous works, this paper integrates low rank approximation and regularization into the training process. We propose Trained Rank Pruning (TRP), which alternates between low rank approximation and training. TRP maintains the capacity of the original network while imposing low-rank constraints during training. A nuclear regularization optimized by stochastic sub-gradient descent is utilized to further promote low rank in TRP. The TRP trained network inherently has a low-rank structure, and is approximated with negligible performance loss, thus eliminating the fine-tuning process after low rank decomposition. The proposed method is comprehensively evaluated on CIFAR-10 and ImageNet, outperforming previous compression methods using low rank approximation.","url_abs":"https://arxiv.org/abs/2004.14566v1","url_pdf":"https://arxiv.org/pdf/2004.14566v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"trp-trained-rank-pruning-for-efficient-deep","repo_url":"https://github.com/yuhuixu1993/Trained-Rank-Pruning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2004.14566","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2004.14566"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yuhuixu1993/Trained-Rank-Pruning","reach":null}],"summary":{"ran":2,"unverified":2},"by_repo_kind":{"listed":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"d1b8a0724e607c78","entry":"MultiPathConv","repo":"yuhuixu1993/Trained-Rank-Pruning","repo_kind":"listed","path":"decompose.py","file_url":"https://github.com/yuhuixu1993/Trained-Rank-Pruning/blob/HEAD/decompose.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d1b8a0724e607c78"}},{"code_sha256_prefix":"b8c55d7ef22663ad","entry":"pd_conv","repo":"yuhuixu1993/Trained-Rank-Pruning","repo_kind":"listed","path":"decompose.py","file_url":"https://github.com/yuhuixu1993/Trained-Rank-Pruning/blob/HEAD/decompose.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b8c55d7ef22663ad"}},{"code_sha256_prefix":"8e94c3ab598d601d","entry":"_set_model_attr","repo":"yuhuixu1993/Trained-Rank-Pruning","repo_kind":"listed","path":"decompose.py","file_url":"https://github.com/yuhuixu1993/Trained-Rank-Pruning/blob/HEAD/decompose.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8e94c3ab598d601d"}},{"code_sha256_prefix":"0a1b4ae9f13aec4f","entry":"network_decouple","repo":"yuhuixu1993/Trained-Rank-Pruning","repo_kind":"listed","path":"decompose.py","file_url":"https://github.com/yuhuixu1993/Trained-Rank-Pruning/blob/HEAD/decompose.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0a1b4ae9f13aec4f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}