{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/prunetrain-gradual-structured-pruning-from","title":"PruneTrain: Fast Neural Network Training by Dynamic Sparse Model Reconfiguration","arxiv_id":"1901.09290","date":"2019-01-26","proceeding":null,"authors":["Sangkug Lym","Esha Choukse","Siavash Zangeneh","Wei Wen","Sujay Sanghavi","Mattan Erez"],"abstract":"State-of-the-art convolutional neural networks (CNNs) used in vision applications have large models with numerous weights. Training these models is very compute- and memory-resource intensive. Much research has been done on pruning or compressing these models to reduce the cost of inference, but little work has addressed the costs of training. We focus precisely on accelerating training. We propose PruneTrain, a cost-efficient mechanism that gradually reduces the training cost during training. PruneTrain uses a structured group-lasso regularization approach that drives the training optimization toward both high accuracy and small weight values. Small weights can then be periodically removed by reconfiguring the network model to a smaller one. By using a structured-pruning approach and additional reconfiguration techniques we introduce, the pruned model can still be efficiently processed on a GPU accelerator. Overall, PruneTrain achieves a reduction of 39% in the end-to-end training time of ResNet50 for ImageNet by reducing computation cost by 40% in FLOPs, memory accesses by 37% for memory bandwidth bound layers, and the inter-accelerator communication by 55%.","url_abs":"https://arxiv.org/abs/1901.09290v5","url_pdf":"https://arxiv.org/pdf/1901.09290v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"prunetrain-gradual-structured-pruning-from","repo_url":"https://github.com/VictorSuciu/prunetrain","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":null,"task_name":"GPU"}],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1901.09290","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1901.09290"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/VictorSuciu/prunetrain","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":5},"by_repo_kind":{"listed":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e04cc6227c28794a","entry":"getRmLayers","repo":"VictorSuciu/prunetrain","repo_kind":"listed","path":"src/custom/rm_layers.py","file_url":"https://github.com/VictorSuciu/prunetrain/blob/HEAD/src/custom/rm_layers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e04cc6227c28794a"}},{"code_sha256_prefix":"0a76202346a9cc6f","entry":"get_group_lasso_global","repo":"VictorSuciu/prunetrain","repo_kind":"listed","path":"src/custom/group_lasso_regs.py","file_url":"https://github.com/VictorSuciu/prunetrain/blob/HEAD/src/custom/group_lasso_regs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0a76202346a9cc6f"}},{"code_sha256_prefix":"c02f7d1d36b91695","entry":"get_group_lasso_group","repo":"VictorSuciu/prunetrain","repo_kind":"listed","path":"src/custom/group_lasso_regs.py","file_url":"https://github.com/VictorSuciu/prunetrain/blob/HEAD/src/custom/group_lasso_regs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c02f7d1d36b91695"}},{"code_sha256_prefix":"771086dc2d149726","entry":"n","repo":"VictorSuciu/prunetrain","repo_kind":"listed","path":"src/custom/resnet_stages.py","file_url":"https://github.com/VictorSuciu/prunetrain/blob/HEAD/src/custom/resnet_stages.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"771086dc2d149726"}},{"code_sha256_prefix":"ea8a8a789e917c32","entry":"third_largest","repo":"VictorSuciu/prunetrain","repo_kind":"listed","path":"src/custom/checkpoint_utils.py","file_url":"https://github.com/VictorSuciu/prunetrain/blob/HEAD/src/custom/checkpoint_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ea8a8a789e917c32"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}