{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/preventing-catastrophic-overfitting-in-fast","title":"Preventing Catastrophic Overfitting in Fast Adversarial Training: A Bi-level Optimization Perspective","arxiv_id":"2407.12443","date":"2024-07-17","proceeding":null,"authors":["Zhaoxin Wang","Handing Wang","Cong Tian","Yaochu Jin"],"abstract":"Adversarial training (AT) has become an effective defense method against adversarial examples (AEs) and it is typically framed as a bi-level optimization problem. Among various AT methods, fast AT (FAT), which employs a single-step attack strategy to guide the training process, can achieve good robustness against adversarial attacks at a low cost. However, FAT methods suffer from the catastrophic overfitting problem, especially on complex tasks or with large-parameter models. In this work, we propose a FAT method termed FGSM-PCO, which mitigates catastrophic overfitting by averting the collapse of the inner optimization problem in the bi-level optimization process. FGSM-PCO generates current-stage AEs from the historical AEs and incorporates them into the training process using an adaptive mechanism. This mechanism determines an appropriate fusion ratio according to the performance of the AEs on the training model. Coupled with a loss function tailored to the training framework, FGSM-PCO can alleviate catastrophic overfitting and help the recovery of an overfitted model to effective training. We evaluate our algorithm across three models and three datasets to validate its effectiveness. Comparative empirical studies against other FAT algorithms demonstrate that our proposed method effectively addresses unresolved overfitting issues in existing algorithms.","url_abs":"https://arxiv.org/abs/2407.12443v1","url_pdf":"https://arxiv.org/pdf/2407.12443v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"preventing-catastrophic-overfitting-in-fast","repo_url":"https://github.com/HandingWangXDGroup/FGSM-PCO","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.12443","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.12443"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/HandingWangXDGroup/FGSM-PCO","reach":null}],"summary":{"ran_fixture":1,"unverified":4},"by_repo_kind":{"official":{"samples":5,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"eda67bbf007849b8","entry":"atta_aug","repo":"HandingWangXDGroup/FGSM-PCO","repo_kind":"official","path":"FGSM-PCO.py","file_url":"https://github.com/HandingWangXDGroup/FGSM-PCO/blob/HEAD/FGSM-PCO.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eda67bbf007849b8"}},{"code_sha256_prefix":"abd0cd6b28aa41cb","entry":"LabelSmoothLoss","repo":"HandingWangXDGroup/FGSM-PCO","repo_kind":"official","path":"FGSM-PCO.py","file_url":"https://github.com/HandingWangXDGroup/FGSM-PCO/blob/HEAD/FGSM-PCO.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"abd0cd6b28aa41cb"}},{"code_sha256_prefix":"c151b506126b02e5","entry":"_label_smoothing","repo":"HandingWangXDGroup/FGSM-PCO","repo_kind":"official","path":"FGSM-PCO.py","file_url":"https://github.com/HandingWangXDGroup/FGSM-PCO/blob/HEAD/FGSM-PCO.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c151b506126b02e5"}},{"code_sha256_prefix":"b3fae58feea5a38b","entry":"get_args","repo":"HandingWangXDGroup/FGSM-PCO","repo_kind":"official","path":"FGSM-PCO.py","file_url":"https://github.com/HandingWangXDGroup/FGSM-PCO/blob/HEAD/FGSM-PCO.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b3fae58feea5a38b"}},{"code_sha256_prefix":"9bf5d16916540519","entry":"main","repo":"HandingWangXDGroup/FGSM-PCO","repo_kind":"official","path":"FGSM-PCO.py","file_url":"https://github.com/HandingWangXDGroup/FGSM-PCO/blob/HEAD/FGSM-PCO.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9bf5d16916540519"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}