{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-pruning-adversarially-robust-neural","title":"HYDRA: Pruning Adversarially Robust Neural Networks","arxiv_id":"2002.10509","date":"2020-02-24","proceeding":"NeurIPS 2020 12","authors":["Vikash Sehwag","Shiqi Wang","Prateek Mittal","Suman Jana"],"abstract":"In safety-critical but computationally resource-constrained applications, deep learning faces two key challenges: lack of robustness against adversarial attacks and large neural network size (often millions of parameters). While the research community has extensively explored the use of robust training and network pruning independently to address one of these challenges, only a few recent works have studied them jointly. However, these works inherit a heuristic pruning strategy that was developed for benign training, which performs poorly when integrated with robust training techniques, including adversarial training and verifiable robust training. To overcome this challenge, we propose to make pruning techniques aware of the robust training objective and let the training objective guide the search for which connections to prune. We realize this insight by formulating the pruning objective as an empirical risk minimization problem which is solved efficiently using SGD. We demonstrate that our approach, titled HYDRA, achieves compressed networks with state-of-the-art benign and robust accuracy, simultaneously. We demonstrate the success of our approach across CIFAR-10, SVHN, and ImageNet dataset with four robust training techniques: iterative adversarial training, randomized smoothing, MixTrain, and CROWN-IBP. We also demonstrate the existence of highly robust sub-networks within non-robust networks. Our code and compressed networks are publicly available at \\url{https://github.com/inspire-group/compactness-robustness}.","url_abs":"https://arxiv.org/abs/2002.10509v3","url_pdf":"https://arxiv.org/pdf/2002.10509v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-pruning-adversarially-robust-neural","repo_url":"https://github.com/inspire-group/compactness-robustness","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"on-pruning-adversarially-robust-neural","repo_url":"https://github.com/inspire-group/hydra","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"on-pruning-adversarially-robust-neural","repo_url":"https://github.com/optml-group/bip","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"on-pruning-adversarially-robust-neural","repo_url":"https://github.com/unites-lab/vpns","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"network-pruning","task_name":"Network Pruning"}],"methods":[{"method_slug":"pruning","method_name":"Pruning"},{"method_slug":"sgd","method_name":"SGD"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2002.10509","atlas_url":"https://app.syntology.ai/?focus=2002.10509","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2002.10509"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/optml-group/bip","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/inspire-group/compactness-robustness","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/inspire-group/hydra","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/unites-lab/vpns","reach":null}],"summary":{"ran":4,"unverified":15},"by_repo_kind":{"listed":{"samples":19,"ran":4,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":17,"samples":[{"code_sha256_prefix":"3c96583f581f4d37","entry":"GetSubnet","repo":"unites-lab/vpns","repo_kind":"listed","path":"unstructured_network_with_score.py","file_url":"https://github.com/unites-lab/vpns/blob/HEAD/unstructured_network_with_score.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3c96583f581f4d37"}},{"code_sha256_prefix":"9c06e50d9e8c334e","entry":"GetSubnet","repo":"inspire-group/hydra","repo_kind":"listed","path":"symbolic_interval/symbolic_network.py","file_url":"https://github.com/inspire-group/hydra/blob/HEAD/symbolic_interval/symbolic_network.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9c06e50d9e8c334e"}},{"code_sha256_prefix":"7440b4e5bc3f91bc","entry":"Interval_BN","repo":"inspire-group/hydra","repo_kind":"listed","path":"symbolic_interval/symbolic_network.py","file_url":"https://github.com/inspire-group/hydra/blob/HEAD/symbolic_interval/symbolic_network.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7440b4e5bc3f91bc"}},{"code_sha256_prefix":"9fa88431a218cf84","entry":"SubnetConv","repo":"unites-lab/vpns","repo_kind":"listed","path":"unstructured_network_with_score.py","file_url":"https://github.com/unites-lab/vpns/blob/HEAD/unstructured_network_with_score.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9fa88431a218cf84"}},{"code_sha256_prefix":"95be6efd42f19c4d","entry":"Center_symbolic_interval","repo":"inspire-group/hydra","repo_kind":"listed","path":"symbolic_interval/symbolic_network.py","file_url":"https://github.com/inspire-group/hydra/blob/HEAD/symbolic_interval/symbolic_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"95be6efd42f19c4d"}},{"code_sha256_prefix":"727eb7c9df20157d","entry":"Interval_Bound","repo":"inspire-group/hydra","repo_kind":"listed","path":"symbolic_interval/symbolic_network.py","file_url":"https://github.com/inspire-group/hydra/blob/HEAD/symbolic_interval/symbolic_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"727eb7c9df20157d"}},{"code_sha256_prefix":"c24d64ff812f33fb","entry":"Interval_Conv2d","repo":"inspire-group/hydra","repo_kind":"listed","path":"symbolic_interval/symbolic_network.py","file_url":"https://github.com/inspire-group/hydra/blob/HEAD/symbolic_interval/symbolic_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c24d64ff812f33fb"}},{"code_sha256_prefix":"e3fcced1e96eb0f2","entry":"Interval_Dense","repo":"inspire-group/hydra","repo_kind":"listed","path":"symbolic_interval/symbolic_network.py","file_url":"https://github.com/inspire-group/hydra/blob/HEAD/symbolic_interval/symbolic_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e3fcced1e96eb0f2"}},{"code_sha256_prefix":"001e355f33be54d7","entry":"Interval_Flatten","repo":"inspire-group/hydra","repo_kind":"listed","path":"symbolic_interval/symbolic_network.py","file_url":"https://github.com/inspire-group/hydra/blob/HEAD/symbolic_interval/symbolic_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"001e355f33be54d7"}},{"code_sha256_prefix":"f6c5efd14db6201c","entry":"Interval_ReLU","repo":"inspire-group/hydra","repo_kind":"listed","path":"symbolic_interval/symbolic_network.py","file_url":"https://github.com/inspire-group/hydra/blob/HEAD/symbolic_interval/symbolic_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f6c5efd14db6201c"}},{"code_sha256_prefix":"7b61dbf5d4a8dbd0","entry":"Interval_SubConv","repo":"inspire-group/hydra","repo_kind":"listed","path":"symbolic_interval/symbolic_network.py","file_url":"https://github.com/inspire-group/hydra/blob/HEAD/symbolic_interval/symbolic_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7b61dbf5d4a8dbd0"}},{"code_sha256_prefix":"a65a8d15783b7474","entry":"Interval_SubLinear","repo":"inspire-group/hydra","repo_kind":"listed","path":"symbolic_interval/symbolic_network.py","file_url":"https://github.com/inspire-group/hydra/blob/HEAD/symbolic_interval/symbolic_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a65a8d15783b7474"}},{"code_sha256_prefix":"f108d8edefeb2b08","entry":"Interval_network","repo":"inspire-group/hydra","repo_kind":"listed","path":"symbolic_interval/symbolic_network.py","file_url":"https://github.com/inspire-group/hydra/blob/HEAD/symbolic_interval/symbolic_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f108d8edefeb2b08"}},{"code_sha256_prefix":"f62e45a681df257c","entry":"Inverse_interval","repo":"inspire-group/hydra","repo_kind":"listed","path":"symbolic_interval/symbolic_network.py","file_url":"https://github.com/inspire-group/hydra/blob/HEAD/symbolic_interval/symbolic_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f62e45a681df257c"}},{"code_sha256_prefix":"0ba759110f8efeb7","entry":"Symbolic_interval","repo":"inspire-group/hydra","repo_kind":"listed","path":"symbolic_interval/symbolic_network.py","file_url":"https://github.com/inspire-group/hydra/blob/HEAD/symbolic_interval/symbolic_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0ba759110f8efeb7"}},{"code_sha256_prefix":"01f2161a52449ec4","entry":"Symbolic_interval_proj1","repo":"inspire-group/hydra","repo_kind":"listed","path":"symbolic_interval/symbolic_network.py","file_url":"https://github.com/inspire-group/hydra/blob/HEAD/symbolic_interval/symbolic_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"01f2161a52449ec4"}},{"code_sha256_prefix":"cfe05853a4302ca5","entry":"Symbolic_interval_proj2","repo":"inspire-group/hydra","repo_kind":"listed","path":"symbolic_interval/symbolic_network.py","file_url":"https://github.com/inspire-group/hydra/blob/HEAD/symbolic_interval/symbolic_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cfe05853a4302ca5"}},{"code_sha256_prefix":"c2a80e1b6f39c437","entry":"gen_sym","repo":"inspire-group/hydra","repo_kind":"listed","path":"symbolic_interval/symbolic_network.py","file_url":"https://github.com/inspire-group/hydra/blob/HEAD/symbolic_interval/symbolic_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c2a80e1b6f39c437"}},{"code_sha256_prefix":"9e06a651cf81f00f","entry":"mix_interval","repo":"inspire-group/hydra","repo_kind":"listed","path":"symbolic_interval/symbolic_network.py","file_url":"https://github.com/inspire-group/hydra/blob/HEAD/symbolic_interval/symbolic_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9e06a651cf81f00f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}