{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/self-ensemble-protection-training-checkpoints","title":"Self-Ensemble Protection: Training Checkpoints Are Good Data Protectors","arxiv_id":"2211.12005","date":"2022-11-22","proceeding":null,"authors":["Sizhe Chen","Geng Yuan","Xinwen Cheng","Yifan Gong","Minghai Qin","Yanzhi Wang","Xiaolin Huang"],"abstract":"As data becomes increasingly vital, a company would be very cautious about releasing data, because the competitors could use it to train high-performance models, thereby posing a tremendous threat to the company's commercial competence. To prevent training good models on the data, we could add imperceptible perturbations to it. Since such perturbations aim at hurting the entire training process, they should reflect the vulnerability of DNN training, rather than that of a single model. Based on this new idea, we seek perturbed examples that are always unrecognized (never correctly classified) in training. In this paper, we uncover them by model checkpoints' gradients, forming the proposed self-ensemble protection (SEP), which is very effective because (1) learning on examples ignored during normal training tends to yield DNNs ignoring normal examples; (2) checkpoints' cross-model gradients are close to orthogonal, meaning that they are as diverse as DNNs with different architectures. That is, our amazing performance of ensemble only requires the computation of training one model. By extensive experiments with 9 baselines on 3 datasets and 5 architectures, SEP is verified to be a new state-of-the-art, e.g., our small $\\ell_\\infty=2/255$ perturbations reduce the accuracy of a CIFAR-10 ResNet18 from 94.56% to 14.68%, compared to 41.35% by the best-known method. Code is available at https://github.com/Sizhe-Chen/SEP.","url_abs":"https://arxiv.org/abs/2211.12005v3","url_pdf":"https://arxiv.org/pdf/2211.12005v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"self-ensemble-protection-training-checkpoints","repo_url":"https://github.com/sizhe-chen/sep","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2211.12005","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2211.12005"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Sizhe-Chen/SEP","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sizhe-chen/sep","reach":{"status":"ok"}}],"summary":{"ran_fixture":1,"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":6,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"098344b3a1ff9bea","entry":"compute_correct","repo":"sizhe-chen/sep","repo_kind":"official","path":"ens.py","file_url":"https://github.com/sizhe-chen/sep/blob/HEAD/ens.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"098344b3a1ff9bea"}},{"code_sha256_prefix":"2f4a2990cb7bdbd1","entry":"get_dataloader","repo":"sizhe-chen/sep","repo_kind":"official","path":"vanilla.py","file_url":"https://github.com/sizhe-chen/sep/blob/HEAD/vanilla.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2f4a2990cb7bdbd1"}},{"code_sha256_prefix":"e246a0e4d92b7f70","entry":"get_dataloader","repo":"sizhe-chen/sep","repo_kind":"official","path":"ens.py","file_url":"https://github.com/sizhe-chen/sep/blob/HEAD/ens.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e246a0e4d92b7f70"}},{"code_sha256_prefix":"7c2ff343d57dd678","entry":"get_mean_feature","repo":"sizhe-chen/sep","repo_kind":"official","path":"ens_feature.py","file_url":"https://github.com/sizhe-chen/sep/blob/HEAD/ens_feature.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7c2ff343d57dd678"}},{"code_sha256_prefix":"003d8a66eb7cb80b","entry":"valid_test","repo":"sizhe-chen/sep","repo_kind":"official","path":"ens.py","file_url":"https://github.com/sizhe-chen/sep/blob/HEAD/ens.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"003d8a66eb7cb80b"}},{"code_sha256_prefix":"a58c8d3462fec315","entry":"valid_test","repo":"sizhe-chen/sep","repo_kind":"official","path":"vanilla.py","file_url":"https://github.com/sizhe-chen/sep/blob/HEAD/vanilla.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a58c8d3462fec315"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}