{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-the-first-adversarially-robust-neural","title":"Towards the first adversarially robust neural network model on MNIST","arxiv_id":"1805.09190","date":"2018-05-23","proceeding":"ICLR 2019 5","authors":["Lukas Schott","Jonas Rauber","Matthias Bethge","Wieland Brendel"],"abstract":"Despite much effort, deep neural networks remain highly susceptible to tiny\ninput perturbations and even for MNIST, one of the most common toy datasets in\ncomputer vision, no neural network model exists for which adversarial\nperturbations are large and make semantic sense to humans. We show that even\nthe widely recognized and by far most successful defense by Madry et al. (1)\noverfits on the L-infinity metric (it's highly susceptible to L2 and L0\nperturbations), (2) classifies unrecognizable images with high certainty, (3)\nperforms not much better than simple input binarization and (4) features\nadversarial perturbations that make little sense to humans. These results\nsuggest that MNIST is far from being solved in terms of adversarial robustness.\nWe present a novel robust classification model that performs analysis by\nsynthesis using learned class-conditional data distributions. We derive bounds\non the robustness and go to great length to empirically evaluate our model\nusing maximally effective adversarial attacks by (a) applying decision-based,\nscore-based, gradient-based and transfer-based attacks for several different Lp\nnorms, (b) by designing a new attack that exploits the structure of our\ndefended model and (c) by devising a novel decision-based attack that seeks to\nminimize the number of perturbed pixels (L0). The results suggest that our\napproach yields state-of-the-art robustness on MNIST against L0, L2 and\nL-infinity perturbations and we demonstrate that most adversarial examples are\nstrongly perturbed towards the perceptual boundary between the original and the\nadversarial class.","url_abs":"http://arxiv.org/abs/1805.09190v3","url_pdf":"http://arxiv.org/pdf/1805.09190v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-the-first-adversarially-robust-neural","repo_url":"https://github.com/bethgelab/AnalysisBySynthesis","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"towards-the-first-adversarially-robust-neural","repo_url":"https://github.com/jonasrauber/analysis-by-synthesis","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"towards-the-first-adversarially-robust-neural","repo_url":"https://github.com/lukas-schott/AnalysisBySynthesis","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"adversarial-robustness","task_name":"Adversarial Robustness"},{"task_slug":"binarization","task_name":"Binarization"},{"task_slug":"robust-classification","task_name":"Robust classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1805.09190","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1805.09190"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jonasrauber/analysis-by-synthesis","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bethgelab/AnalysisBySynthesis","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lukas-schott/AnalysisBySynthesis","reach":{"status":"ok"}}],"summary":{"unverified":10},"by_repo_kind":{"listed":{"samples":10,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2c8f5ea9c0e6fd95","entry":"binarize","repo":"bethgelab/AnalysisBySynthesis","repo_kind":"listed","path":"abs_models/utils.py","file_url":"https://github.com/bethgelab/AnalysisBySynthesis/blob/HEAD/abs_models/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2c8f5ea9c0e6fd95"}},{"code_sha256_prefix":"624c2dd50aa62ca1","entry":"calc_fov","repo":"bethgelab/AnalysisBySynthesis","repo_kind":"listed","path":"abs_models/nets.py","file_url":"https://github.com/bethgelab/AnalysisBySynthesis/blob/HEAD/abs_models/nets.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"624c2dd50aa62ca1"}},{"code_sha256_prefix":"3499d7bccd145664","entry":"calc_fov_layer","repo":"bethgelab/AnalysisBySynthesis","repo_kind":"listed","path":"abs_models/nets.py","file_url":"https://github.com/bethgelab/AnalysisBySynthesis/blob/HEAD/abs_models/nets.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3499d7bccd145664"}},{"code_sha256_prefix":"5e9bf57f7805934c","entry":"clip_to_sphere","repo":"bethgelab/AnalysisBySynthesis","repo_kind":"listed","path":"abs_models/utils.py","file_url":"https://github.com/bethgelab/AnalysisBySynthesis/blob/HEAD/abs_models/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5e9bf57f7805934c"}},{"code_sha256_prefix":"da2ea3c124b9c9ab","entry":"fig2img","repo":"bethgelab/AnalysisBySynthesis","repo_kind":"listed","path":"abs_models/visualization.py","file_url":"https://github.com/bethgelab/AnalysisBySynthesis/blob/HEAD/abs_models/visualization.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"da2ea3c124b9c9ab"}},{"code_sha256_prefix":"2a408b92ba2874cc","entry":"get_batch","repo":"bethgelab/AnalysisBySynthesis","repo_kind":"listed","path":"abs_models/utils.py","file_url":"https://github.com/bethgelab/AnalysisBySynthesis/blob/HEAD/abs_models/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2a408b92ba2874cc"}},{"code_sha256_prefix":"e864ed0d2de9f807","entry":"get_gaussian_samples","repo":"bethgelab/AnalysisBySynthesis","repo_kind":"listed","path":"abs_models/sampling.py","file_url":"https://github.com/bethgelab/AnalysisBySynthesis/blob/HEAD/abs_models/sampling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e864ed0d2de9f807"}},{"code_sha256_prefix":"16eefc701124af30","entry":"plot","repo":"bethgelab/AnalysisBySynthesis","repo_kind":"listed","path":"abs_models/visualization.py","file_url":"https://github.com/bethgelab/AnalysisBySynthesis/blob/HEAD/abs_models/visualization.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"16eefc701124af30"}},{"code_sha256_prefix":"6b5f33ffc98f90ec","entry":"update_distal_adv","repo":"bethgelab/AnalysisBySynthesis","repo_kind":"listed","path":"abs_models/attack_utils.py","file_url":"https://github.com/bethgelab/AnalysisBySynthesis/blob/HEAD/abs_models/attack_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6b5f33ffc98f90ec"}},{"code_sha256_prefix":"2d80df353a1615bc","entry":"visualize_image","repo":"bethgelab/AnalysisBySynthesis","repo_kind":"listed","path":"abs_models/visualization.py","file_url":"https://github.com/bethgelab/AnalysisBySynthesis/blob/HEAD/abs_models/visualization.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2d80df353a1615bc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}