{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deepxplore-automated-whitebox-testing-of-deep","title":"DeepXplore: Automated Whitebox Testing of Deep Learning Systems","arxiv_id":"1705.06640","date":"2017-05-18","proceeding":null,"authors":["Kexin Pei","Yinzhi Cao","Junfeng Yang","Suman Jana"],"abstract":"Deep learning (DL) systems are increasingly deployed in safety- and\nsecurity-critical domains including self-driving cars and malware detection,\nwhere the correctness and predictability of a system's behavior for corner case\ninputs are of great importance. Existing DL testing depends heavily on manually\nlabeled data and therefore often fails to expose erroneous behaviors for rare\ninputs.\n  We design, implement, and evaluate DeepXplore, the first whitebox framework\nfor systematically testing real-world DL systems. First, we introduce neuron\ncoverage for systematically measuring the parts of a DL system exercised by\ntest inputs. Next, we leverage multiple DL systems with similar functionality\nas cross-referencing oracles to avoid manual checking. Finally, we demonstrate\nhow finding inputs for DL systems that both trigger many differential behaviors\nand achieve high neuron coverage can be represented as a joint optimization\nproblem and solved efficiently using gradient-based search techniques.\n  DeepXplore efficiently finds thousands of incorrect corner case behaviors\n(e.g., self-driving cars crashing into guard rails and malware masquerading as\nbenign software) in state-of-the-art DL models with thousands of neurons\ntrained on five popular datasets including ImageNet and Udacity self-driving\nchallenge data. For all tested DL models, on average, DeepXplore generated one\ntest input demonstrating incorrect behavior within one second while running\nonly on a commodity laptop. We further show that the test inputs generated by\nDeepXplore can also be used to retrain the corresponding DL model to improve\nthe model's accuracy by up to 3%.","url_abs":"http://arxiv.org/abs/1705.06640v4","url_pdf":"http://arxiv.org/pdf/1705.06640v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deepxplore-automated-whitebox-testing-of-deep","repo_url":"https://github.com/ee17b031-iittp/MNIST-Retrain-with-DeepXplore","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"deepxplore-automated-whitebox-testing-of-deep","repo_url":"https://github.com/owenps/Adversarial_Generator","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"deepxplore-automated-whitebox-testing-of-deep","repo_url":"https://github.com/peikexin9/deepxplore","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"malware-detection","task_name":"Malware Detection"},{"task_slug":"self-driving-cars","task_name":"Self-Driving Cars"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1705.06640","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}