{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/create-and-find-flatness-building-flat","title":"Create and Find Flatness: Building Flat Training Spaces in Advance for Continual Learning","arxiv_id":"2309.11305","date":"2023-09-20","proceeding":null,"authors":["Wenhang Shi","Yiren Chen","Zhe Zhao","Wei Lu","Kimmo Yan","Xiaoyong Du"],"abstract":"Catastrophic forgetting remains a critical challenge in the field of continual learning, where neural networks struggle to retain prior knowledge while assimilating new information. Most existing studies emphasize mitigating this issue only when encountering new tasks, overlooking the significance of the pre-task phase. Therefore, we shift the attention to the current task learning stage, presenting a novel framework, C&F (Create and Find Flatness), which builds a flat training space for each task in advance. Specifically, during the learning of the current task, our framework adaptively creates a flat region around the minimum in the loss landscape. Subsequently, it finds the parameters' importance to the current task based on their flatness degrees. When adapting the model to a new task, constraints are applied according to the flatness and a flat space is simultaneously prepared for the impending task. We theoretically demonstrate the consistency between the created and found flatness. In this manner, our framework not only accommodates ample parameter space for learning new tasks but also preserves the preceding knowledge of earlier tasks. Experimental results exhibit C&F's state-of-the-art performance as a standalone continual learning approach and its efficacy as a framework incorporating other methods. Our work is available at https://github.com/Eric8932/Create-and-Find-Flatness.","url_abs":"https://arxiv.org/abs/2309.11305v1","url_pdf":"https://arxiv.org/pdf/2309.11305v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"create-and-find-flatness-building-flat","repo_url":"https://github.com/Eric8932/Create-and-Find-Flatness","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2309.11305","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.11305"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Eric8932/Create-and-Find-Flatness","reach":null}],"summary":{"ran_fixture":1,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"86a49564506caab3","entry":"remove_dup","repo":"Eric8932/Create-and-Find-Flatness","repo_kind":"official","path":"finetune/run_cls_cf.py","file_url":"https://github.com/Eric8932/Create-and-Find-Flatness/blob/HEAD/finetune/run_cls_cf.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"86a49564506caab3"}},{"code_sha256_prefix":"79a8b57817a41e76","entry":"read_dataset","repo":"Eric8932/Create-and-Find-Flatness","repo_kind":"official","path":"finetune/run_cls_cf.py","file_url":"https://github.com/Eric8932/Create-and-Find-Flatness/blob/HEAD/finetune/run_cls_cf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"79a8b57817a41e76"}},{"code_sha256_prefix":"1adf3a7e375435c3","entry":"sample_dataset","repo":"Eric8932/Create-and-Find-Flatness","repo_kind":"official","path":"finetune/run_cls_cf.py","file_url":"https://github.com/Eric8932/Create-and-Find-Flatness/blob/HEAD/finetune/run_cls_cf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1adf3a7e375435c3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}