{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/comparing-rewinding-and-fine-tuning-in-neural","title":"Comparing Rewinding and Fine-tuning in Neural Network Pruning","arxiv_id":"2003.02389","date":"2020-03-05","proceeding":"ICLR 2020 1","authors":["Alex Renda","Jonathan Frankle","Michael Carbin"],"abstract":"Many neural network pruning algorithms proceed in three steps: train the network to completion, remove unwanted structure to compress the network, and retrain the remaining structure to recover lost accuracy. The standard retraining technique, fine-tuning, trains the unpruned weights from their final trained values using a small fixed learning rate. In this paper, we compare fine-tuning to alternative retraining techniques. Weight rewinding (as proposed by Frankle et al., (2019)), rewinds unpruned weights to their values from earlier in training and retrains them from there using the original training schedule. Learning rate rewinding (which we propose) trains the unpruned weights from their final values using the same learning rate schedule as weight rewinding. Both rewinding techniques outperform fine-tuning, forming the basis of a network-agnostic pruning algorithm that matches the accuracy and compression ratios of several more network-specific state-of-the-art techniques.","url_abs":"https://arxiv.org/abs/2003.02389v1","url_pdf":"https://arxiv.org/pdf/2003.02389v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"comparing-rewinding-and-fine-tuning-in-neural","repo_url":"https://github.com/lottery-ticket/rewinding-iclr20-public","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null},{"paper_slug":"comparing-rewinding-and-fine-tuning-in-neural","repo_url":"https://github.com/gahaalt/reproducing-comparing-rewinding-and-finetuning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":"network-pruning","task_name":"Network Pruning"}],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2003.02389","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2003.02389"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lottery-ticket/rewinding-iclr20-public","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gahaalt/reproducing-comparing-rewinding-and-finetuning","reach":{"status":"ok"}}],"summary":{"ran_honours":2,"ran_draft_wrong":4},"by_repo_kind":{"official":{"samples":6,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"4bb442f2ca92f56b","entry":"aggregate_trials","repo":"lottery-ticket/rewinding-iclr20-public","repo_kind":"official","path":"vision/plot/common.py","file_url":"https://github.com/lottery-ticket/rewinding-iclr20-public/blob/HEAD/vision/plot/common.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4bb442f2ca92f56b"}},{"code_sha256_prefix":"eb9b5f8723fa9304","entry":"bias_name_of_base_name","repo":"lottery-ticket/rewinding-iclr20-public","repo_kind":"official","path":"gnmt/model/nmt/lottery/lottery.py","file_url":"https://github.com/lottery-ticket/rewinding-iclr20-public/blob/HEAD/gnmt/model/nmt/lottery/lottery.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eb9b5f8723fa9304"}},{"code_sha256_prefix":"617c0419b06f337d","entry":"fmt_of_method","repo":"lottery-ticket/rewinding-iclr20-public","repo_kind":"official","path":"vision/plot/common.py","file_url":"https://github.com/lottery-ticket/rewinding-iclr20-public/blob/HEAD/vision/plot/common.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"617c0419b06f337d"}},{"code_sha256_prefix":"ad3ffba2b9c58c92","entry":"mask_name_of_base_name","repo":"lottery-ticket/rewinding-iclr20-public","repo_kind":"official","path":"gnmt/model/nmt/lottery/lottery.py","file_url":"https://github.com/lottery-ticket/rewinding-iclr20-public/blob/HEAD/gnmt/model/nmt/lottery/lottery.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ad3ffba2b9c58c92"}},{"code_sha256_prefix":"c5ac62e39a802ae4","entry":"ms_of_method","repo":"lottery-ticket/rewinding-iclr20-public","repo_kind":"official","path":"vision/plot/common.py","file_url":"https://github.com/lottery-ticket/rewinding-iclr20-public/blob/HEAD/vision/plot/common.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c5ac62e39a802ae4"}},{"code_sha256_prefix":"f40e4115e09f1b77","entry":"weight_name_of_base_name","repo":"lottery-ticket/rewinding-iclr20-public","repo_kind":"official","path":"gnmt/model/nmt/lottery/lottery.py","file_url":"https://github.com/lottery-ticket/rewinding-iclr20-public/blob/HEAD/gnmt/model/nmt/lottery/lottery.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f40e4115e09f1b77"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}