{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-statistical-approach-to-assessing-neural","title":"A Statistical Approach to Assessing Neural Network Robustness","arxiv_id":"1811.07209","date":"2018-11-17","proceeding":"ICLR 2019 5","authors":["Stefan Webb","Tom Rainforth","Yee Whye Teh","M. Pawan Kumar"],"abstract":"We present a new approach to assessing the robustness of neural networks\nbased on estimating the proportion of inputs for which a property is violated.\nSpecifically, we estimate the probability of the event that the property is\nviolated under an input model. Our approach critically varies from the formal\nverification framework in that when the property can be violated, it provides\nan informative notion of how robust the network is, rather than just the\nconventional assertion that the network is not verifiable. Furthermore, it\nprovides an ability to scale to larger networks than formal verification\napproaches. Though the framework still provides a formal guarantee of\nsatisfiability whenever it successfully finds one or more violations, these\nadvantages do come at the cost of only providing a statistical estimate of\nunsatisfiability whenever no violation is found. Key to the practical success\nof our approach is an adaptation of multi-level splitting, a Monte Carlo\napproach for estimating the probability of rare events, to our statistical\nrobustness framework. We demonstrate that our approach is able to emulate\nformal verification procedures on benchmark problems, while scaling to larger\nnetworks and providing reliable additional information in the form of accurate\nestimates of the violation probability.","url_abs":"http://arxiv.org/abs/1811.07209v4","url_pdf":"http://arxiv.org/pdf/1811.07209v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-statistical-approach-to-assessing-neural","repo_url":"https://github.com/oval-group/statistical-robustness","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1811.07209","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1811.07209"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/oval-group/statistical-robustness","reach":null}],"summary":{"ran_violates":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"a9cd3385008cabd7","entry":"cm2inch","repo":"oval-group/statistical-robustness","repo_kind":"official","path":"exp_6_2/run_exp.py","file_url":"https://github.com/oval-group/statistical-robustness/blob/HEAD/exp_6_2/run_exp.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a9cd3385008cabd7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}