{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/reliable-adversarial-distillation-with","title":"Reliable Adversarial Distillation with Unreliable Teachers","arxiv_id":"2106.04928","date":"2021-06-09","proceeding":"ICLR 2022 4","authors":["Jianing Zhu","Jiangchao Yao","Bo Han","Jingfeng Zhang","Tongliang Liu","Gang Niu","Jingren Zhou","Jianliang Xu","Hongxia Yang"],"abstract":"In ordinary distillation, student networks are trained with soft labels (SLs) given by pretrained teacher networks, and students are expected to improve upon teachers since SLs are stronger supervision than the original hard labels. However, when considering adversarial robustness, teachers may become unreliable and adversarial distillation may not work: teachers are pretrained on their own adversarial data, and it is too demanding to require that teachers are also good at every adversarial data queried by students. Therefore, in this paper, we propose reliable introspective adversarial distillation (IAD) where students partially instead of fully trust their teachers. Specifically, IAD distinguishes between three cases given a query of a natural data (ND) and the corresponding adversarial data (AD): (a) if a teacher is good at AD, its SL is fully trusted; (b) if a teacher is good at ND but not AD, its SL is partially trusted and the student also takes its own SL into account; (c) otherwise, the student only relies on its own SL. Experiments demonstrate the effectiveness of IAD for improving upon teachers in terms of adversarial robustness.","url_abs":"https://arxiv.org/abs/2106.04928v3","url_pdf":"https://arxiv.org/pdf/2106.04928v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"reliable-adversarial-distillation-with","repo_url":"https://github.com/zfancy/iad","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"reliable-adversarial-distillation-with","repo_url":"https://github.com/boyellow/adaad","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"adversarial-robustness","task_name":"Adversarial Robustness"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.04928","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.04928"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zfancy/iad","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/boyellow/adaad","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":6},"by_repo_kind":{"official":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6c62112efeaf8abe","entry":"PGD","repo":"zfancy/iad","repo_kind":"official","path":"attack_generator.py","file_url":"https://github.com/zfancy/iad/blob/HEAD/attack_generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6c62112efeaf8abe"}},{"code_sha256_prefix":"d92ef68ca445d801","entry":"ResNet18","repo":"zfancy/iad","repo_kind":"official","path":"models/resnet.py","file_url":"https://github.com/zfancy/iad/blob/HEAD/models/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d92ef68ca445d801"}},{"code_sha256_prefix":"9aac0eec28d31210","entry":"ResNet34","repo":"zfancy/iad","repo_kind":"official","path":"models/resnet.py","file_url":"https://github.com/zfancy/iad/blob/HEAD/models/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9aac0eec28d31210"}},{"code_sha256_prefix":"fbee4d09475f9bc3","entry":"ResNet50","repo":"zfancy/iad","repo_kind":"official","path":"models/resnet.py","file_url":"https://github.com/zfancy/iad/blob/HEAD/models/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fbee4d09475f9bc3"}},{"code_sha256_prefix":"d2682eaa7c907946","entry":"cwloss","repo":"zfancy/iad","repo_kind":"official","path":"attack_generator.py","file_url":"https://github.com/zfancy/iad/blob/HEAD/attack_generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d2682eaa7c907946"}},{"code_sha256_prefix":"4f81fb9d82756432","entry":"eval_clean","repo":"zfancy/iad","repo_kind":"official","path":"attack_generator.py","file_url":"https://github.com/zfancy/iad/blob/HEAD/attack_generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4f81fb9d82756432"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}