{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/trs-transferability-reduced-ensemble-via","title":"TRS: Transferability Reduced Ensemble via Encouraging Gradient Diversity and Model Smoothness","arxiv_id":"2104.00671","date":"2021-04-01","proceeding":"NeurIPS 2021 12","authors":["Zhuolin Yang","Linyi Li","Xiaojun Xu","Shiliang Zuo","Qian Chen","Benjamin Rubinstein","Pan Zhou","Ce Zhang","Bo Li"],"abstract":"Adversarial Transferability is an intriguing property - adversarial perturbation crafted against one model is also effective against another model, while these models are from different model families or training processes. To better protect ML systems against adversarial attacks, several questions are raised: what are the sufficient conditions for adversarial transferability and how to bound it? Is there a way to reduce the adversarial transferability in order to improve the robustness of an ensemble ML model? To answer these questions, in this work we first theoretically analyze and outline sufficient conditions for adversarial transferability between models; then propose a practical algorithm to reduce the transferability between base models within an ensemble to improve its robustness. Our theoretical analysis shows that only promoting the orthogonality between gradients of base models is not enough to ensure low transferability; in the meantime, the model smoothness is an important factor to control the transferability. We also provide the lower and upper bounds of adversarial transferability under certain conditions. Inspired by our theoretical analysis, we propose an effective Transferability Reduced Smooth(TRS) ensemble training strategy to train a robust ensemble with low transferability by enforcing both gradient orthogonality and model smoothness between base models. We conduct extensive experiments on TRS and compare with 6 state-of-the-art ensemble baselines against 8 whitebox attacks on different datasets, demonstrating that the proposed TRS outperforms all baselines significantly.","url_abs":"https://arxiv.org/abs/2104.00671v2","url_pdf":"https://arxiv.org/pdf/2104.00671v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"trs-transferability-reduced-ensemble-via","repo_url":"https://github.com/AI-secure/Transferability-Reduced-Smooth-Ensemble","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2104.00671","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2104.00671"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/AI-secure/Transferability-Reduced-Smooth-Ensemble","reach":null}],"summary":{"ran_draft_wrong":3,"ran_honours":2,"ran":1,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1},"community":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"5f27b8e9e500314b","entry":"get_acc","repo":"yuxiaochen1103/Hi-TRS","repo_kind":"community","path":"train_on_NTU_AR.py","file_url":"https://github.com/yuxiaochen1103/Hi-TRS/blob/HEAD/train_on_NTU_AR.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5f27b8e9e500314b"}},{"code_sha256_prefix":"c5b926bdf355c69d","entry":"Cosine","repo":"AI-secure/Transferability-Reduced-Smooth-Ensemble","repo_kind":"official","path":"train/Empirical/trainer.py","file_url":"https://github.com/AI-secure/Transferability-Reduced-Smooth-Ensemble/blob/HEAD/train/Empirical/trainer.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c5b926bdf355c69d"}},{"code_sha256_prefix":"26fd974a366fee7c","entry":"Ensemble","repo":"AI-secure/Transferability-Reduced-Smooth-Ensemble","repo_kind":"official","path":"train/Empirical/trainer.py","file_url":"https://github.com/AI-secure/Transferability-Reduced-Smooth-Ensemble/blob/HEAD/train/Empirical/trainer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"26fd974a366fee7c"}},{"code_sha256_prefix":"d7d02143411ad4d7","entry":"Magnitude","repo":"AI-secure/Transferability-Reduced-Smooth-Ensemble","repo_kind":"official","path":"train/Empirical/trainer.py","file_url":"https://github.com/AI-secure/Transferability-Reduced-Smooth-Ensemble/blob/HEAD/train/Empirical/trainer.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d7d02143411ad4d7"}},{"code_sha256_prefix":"55b27593beb14f70","entry":"PGD","repo":"AI-secure/Transferability-Reduced-Smooth-Ensemble","repo_kind":"official","path":"train/Empirical/trainer.py","file_url":"https://github.com/AI-secure/Transferability-Reduced-Smooth-Ensemble/blob/HEAD/train/Empirical/trainer.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"55b27593beb14f70"}},{"code_sha256_prefix":"a1d3eaa48d595276","entry":"get_acc","repo":"yuxiaochen1103/Hi-TRS","repo_kind":"community","path":"pretrain_on_NTU.py","file_url":"https://github.com/yuxiaochen1103/Hi-TRS/blob/HEAD/pretrain_on_NTU.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a1d3eaa48d595276"}},{"code_sha256_prefix":"fcc8bbcd4177bb70","entry":"TRS_Trainer","repo":"AI-secure/Transferability-Reduced-Smooth-Ensemble","repo_kind":"official","path":"train/Empirical/trainer.py","file_url":"https://github.com/AI-secure/Transferability-Reduced-Smooth-Ensemble/blob/HEAD/train/Empirical/trainer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fcc8bbcd4177bb70"}},{"code_sha256_prefix":"0122889949b64757","entry":"requires_grad_","repo":"AI-secure/Transferability-Reduced-Smooth-Ensemble","repo_kind":"official","path":"train/Empirical/trainer.py","file_url":"https://github.com/AI-secure/Transferability-Reduced-Smooth-Ensemble/blob/HEAD/train/Empirical/trainer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0122889949b64757"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}