{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/skip-connections-matter-on-the","title":"Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets","arxiv_id":"2002.05990","date":"2020-02-14","proceeding":"ICLR 2020 1","authors":["Dongxian Wu","Yisen Wang","Shu-Tao Xia","James Bailey","Xingjun Ma"],"abstract":"Skip connections are an essential component of current state-of-the-art deep neural networks (DNNs) such as ResNet, WideResNet, DenseNet, and ResNeXt. Despite their huge success in building deeper and more powerful DNNs, we identify a surprising security weakness of skip connections in this paper. Use of skip connections allows easier generation of highly transferable adversarial examples. Specifically, in ResNet-like (with skip connections) neural networks, gradients can backpropagate through either skip connections or residual modules. We find that using more gradients from the skip connections rather than the residual modules according to a decay factor, allows one to craft adversarial examples with high transferability. Our method is termed Skip Gradient Method(SGM). We conduct comprehensive transfer attacks against state-of-the-art DNNs including ResNets, DenseNets, Inceptions, Inception-ResNet, Squeeze-and-Excitation Network (SENet) and robustly trained DNNs. We show that employing SGM on the gradient flow can greatly improve the transferability of crafted attacks in almost all cases. Furthermore, SGM can be easily combined with existing black-box attack techniques, and obtain high improvements over state-of-the-art transferability methods. Our findings not only motivate new research into the architectural vulnerability of DNNs, but also open up further challenges for the design of secure DNN architectures.","url_abs":"https://arxiv.org/abs/2002.05990v1","url_pdf":"https://arxiv.org/pdf/2002.05990v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"skip-connections-matter-on-the","repo_url":"https://github.com/csdongxian/skip-connections-matter","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"skip-connections-matter-on-the","repo_url":"https://github.com/Trustworthy-AI-Group/TransferAttack","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"skip-connections-matter-on-the","repo_url":"https://github.com/csdongxian/csdongxian","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"skip-connections-matter-on-the","repo_url":"https://github.com/mo666666/sgm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"auxiliary-classifier","method_name":"Auxiliary Classifier"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"concatenated-skip-connection","method_name":"Concatenated Skip Connection"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-block","method_name":"Dense Block"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"grouped-convolution","method_name":"Grouped Convolution"},{"method_slug":"inception-module","method_name":"Inception Module"},{"method_slug":"inception-v2","method_name":"Inception v2"},{"method_slug":"inception-resnet-v2","method_name":"Inception-ResNet-v2"},{"method_slug":"inception-resnet-v2-reduction-b","method_name":"Inception-ResNet-v2 Reduction-B"},{"method_slug":"inception-resnet-v2-a","method_name":"Inception-ResNet-v2-A"},{"method_slug":"inception-resnet-v2-b","method_name":"Inception-ResNet-v2-B"},{"method_slug":"inception-resnet-v2-c","method_name":"Inception-ResNet-v2-C"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"reduction-a","method_name":"Reduction-A"},{"method_slug":"resnext","method_name":"ResNeXt"},{"method_slug":"resnext-block","method_name":"ResNeXt Block"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"senet","method_name":"SENet"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"squeeze-and-excitation-block","method_name":"Squeeze-and-Excitation Block"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2002.05990","atlas_url":"https://app.syntology.ai/?focus=2002.05990","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2002.05990"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Trustworthy-AI-Group/TransferAttack","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/csdongxian/skip-connections-matter","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mo666666/sgm","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/csdongxian/csdongxian","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":3,"unverified":6},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1},"listed":{"samples":6,"ran":3,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1a88c8d0cf565ac1","entry":"backward_hook","repo":"mo666666/sgm","repo_kind":"listed","path":"utils_sgm.py","file_url":"https://github.com/mo666666/sgm/blob/HEAD/utils_sgm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1a88c8d0cf565ac1"}},{"code_sha256_prefix":"89676b593803f1d1","entry":"backward_hook","repo":"Trustworthy-AI-Group/TransferAttack","repo_kind":"listed","path":"transferattack/model_related/sgm.py","file_url":"https://github.com/Trustworthy-AI-Group/TransferAttack/blob/HEAD/transferattack/model_related/sgm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"89676b593803f1d1"}},{"code_sha256_prefix":"3e47156aa6454022","entry":"backward_hook_norm","repo":"Trustworthy-AI-Group/TransferAttack","repo_kind":"listed","path":"transferattack/model_related/sgm.py","file_url":"https://github.com/Trustworthy-AI-Group/TransferAttack/blob/HEAD/transferattack/model_related/sgm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e47156aa6454022"}},{"code_sha256_prefix":"d12f0c422c2de806","entry":"SGM","repo":"Trustworthy-AI-Group/TransferAttack","repo_kind":"listed","path":"transferattack/model_related/sgm.py","file_url":"https://github.com/Trustworthy-AI-Group/TransferAttack/blob/HEAD/transferattack/model_related/sgm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d12f0c422c2de806"}},{"code_sha256_prefix":"f0c9a29156911331","entry":"accuracy","repo":"csdongxian/skip-connections-matter","repo_kind":"official","path":"evaluate.py","file_url":"https://github.com/csdongxian/skip-connections-matter/blob/HEAD/evaluate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f0c9a29156911331"}},{"code_sha256_prefix":"35b0c8af636d7d79","entry":"backward_hook_norm","repo":"csdongxian/skip-connections-matter","repo_kind":"official","path":"utils_sgm.py","file_url":"https://github.com/csdongxian/skip-connections-matter/blob/HEAD/utils_sgm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"35b0c8af636d7d79"}},{"code_sha256_prefix":"1e0f16878c9ad4d6","entry":"register_hook_for_densenet","repo":"Trustworthy-AI-Group/TransferAttack","repo_kind":"listed","path":"transferattack/model_related/sgm.py","file_url":"https://github.com/Trustworthy-AI-Group/TransferAttack/blob/HEAD/transferattack/model_related/sgm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1e0f16878c9ad4d6"}},{"code_sha256_prefix":"5f12591f00431049","entry":"register_hook_for_resnet","repo":"Trustworthy-AI-Group/TransferAttack","repo_kind":"listed","path":"transferattack/model_related/sgm.py","file_url":"https://github.com/Trustworthy-AI-Group/TransferAttack/blob/HEAD/transferattack/model_related/sgm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5f12591f00431049"}},{"code_sha256_prefix":"55f8205825bac4f7","entry":"validate","repo":"csdongxian/skip-connections-matter","repo_kind":"official","path":"evaluate.py","file_url":"https://github.com/csdongxian/skip-connections-matter/blob/HEAD/evaluate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"55f8205825bac4f7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}