{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/eliminating-catastrophic-overfitting-via-1","title":"Eliminating Catastrophic Overfitting Via Abnormal Adversarial Examples Regularization","arxiv_id":"2404.08154","date":"2024-04-11","proceeding":"NeurIPS 2023 11","authors":["Runqi Lin","Chaojian Yu","Tongliang Liu"],"abstract":"Single-step adversarial training (SSAT) has demonstrated the potential to achieve both efficiency and robustness. However, SSAT suffers from catastrophic overfitting (CO), a phenomenon that leads to a severely distorted classifier, making it vulnerable to multi-step adversarial attacks. In this work, we observe that some adversarial examples generated on the SSAT-trained network exhibit anomalous behaviour, that is, although these training samples are generated by the inner maximization process, their associated loss decreases instead, which we named abnormal adversarial examples (AAEs). Upon further analysis, we discover a close relationship between AAEs and classifier distortion, as both the number and outputs of AAEs undergo a significant variation with the onset of CO. Given this observation, we re-examine the SSAT process and uncover that before the occurrence of CO, the classifier already displayed a slight distortion, indicated by the presence of few AAEs. Furthermore, the classifier directly optimizing these AAEs will accelerate its distortion, and correspondingly, the variation of AAEs will sharply increase as a result. In such a vicious circle, the classifier rapidly becomes highly distorted and manifests as CO within a few iterations. These observations motivate us to eliminate CO by hindering the generation of AAEs. Specifically, we design a novel method, termed Abnormal Adversarial Examples Regularization (AAER), which explicitly regularizes the variation of AAEs to hinder the classifier from becoming distorted. Extensive experiments demonstrate that our method can effectively eliminate CO and further boost adversarial robustness with negligible additional computational overhead.","url_abs":"https://arxiv.org/abs/2404.08154v2","url_pdf":"https://arxiv.org/pdf/2404.08154v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"eliminating-catastrophic-overfitting-via-1","repo_url":"https://github.com/tmllab/2023_neurips_aaer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"adversarial-robustness","task_name":"Adversarial Robustness"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2404.08154","atlas_url":"https://app.syntology.ai/?focus=2404.08154","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.08154"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/tmllab/2023_NeurIPS_AAER","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tmllab/2023_neurips_aaer","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"ran_violates":1},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0234f49f9592dec8","entry":"PreActResNet18","repo":"tmllab/2023_neurips_aaer","repo_kind":"official","path":"CIFAR10/preact_resnet.py","file_url":"https://github.com/tmllab/2023_neurips_aaer/blob/HEAD/CIFAR10/preact_resnet.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0234f49f9592dec8"}},{"code_sha256_prefix":"403277d0bfbb477b","entry":"find_classes","repo":"tmllab/2023_neurips_aaer","repo_kind":"official","path":"Imagenet-100/Imagenet.py","file_url":"https://github.com/tmllab/2023_neurips_aaer/blob/HEAD/Imagenet-100/Imagenet.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"403277d0bfbb477b"}},{"code_sha256_prefix":"ab4109634b75ef8b","entry":"is_image_file","repo":"tmllab/2023_neurips_aaer","repo_kind":"official","path":"Imagenet-100/Imagenet.py","file_url":"https://github.com/tmllab/2023_neurips_aaer/blob/HEAD/Imagenet-100/Imagenet.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ab4109634b75ef8b"}},{"code_sha256_prefix":"c5c0f6aba61fc840","entry":"l2_square","repo":"tmllab/2023_neurips_aaer","repo_kind":"official","path":"Imagenet-100/utils.py","file_url":"https://github.com/tmllab/2023_neurips_aaer/blob/HEAD/Imagenet-100/utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c5c0f6aba61fc840"}},{"code_sha256_prefix":"27e7954cc9b19461","entry":"loadPILImage","repo":"tmllab/2023_neurips_aaer","repo_kind":"official","path":"Imagenet-100/Imagenet.py","file_url":"https://github.com/tmllab/2023_neurips_aaer/blob/HEAD/Imagenet-100/Imagenet.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"27e7954cc9b19461"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}