{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/practical-black-box-attacks-against-machine","title":"Practical Black-Box Attacks against Machine Learning","arxiv_id":"1602.02697","date":"2016-02-08","proceeding":null,"authors":["Nicolas Papernot","Patrick McDaniel","Ian Goodfellow","Somesh Jha","Z. Berkay Celik","Ananthram Swami"],"abstract":"Machine learning (ML) models, e.g., deep neural networks (DNNs), are\nvulnerable to adversarial examples: malicious inputs modified to yield\nerroneous model outputs, while appearing unmodified to human observers.\nPotential attacks include having malicious content like malware identified as\nlegitimate or controlling vehicle behavior. Yet, all existing adversarial\nexample attacks require knowledge of either the model internals or its training\ndata. We introduce the first practical demonstration of an attacker controlling\na remotely hosted DNN with no such knowledge. Indeed, the only capability of\nour black-box adversary is to observe labels given by the DNN to chosen inputs.\nOur attack strategy consists in training a local model to substitute for the\ntarget DNN, using inputs synthetically generated by an adversary and labeled by\nthe target DNN. We use the local substitute to craft adversarial examples, and\nfind that they are misclassified by the targeted DNN. To perform a real-world\nand properly-blinded evaluation, we attack a DNN hosted by MetaMind, an online\ndeep learning API. We find that their DNN misclassifies 84.24% of the\nadversarial examples crafted with our substitute. We demonstrate the general\napplicability of our strategy to many ML techniques by conducting the same\nattack against models hosted by Amazon and Google, using logistic regression\nsubstitutes. They yield adversarial examples misclassified by Amazon and Google\nat rates of 96.19% and 88.94%. We also find that this black-box attack strategy\nis capable of evading defense strategies previously found to make adversarial\nexample crafting harder.","url_abs":"http://arxiv.org/abs/1602.02697v4","url_pdf":"http://arxiv.org/pdf/1602.02697v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/AngusG/cleverhans-attacking-bnns","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/HowToMakeABomb101/Hot2MakeAB0mbSite","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/adrian-botta/understanding_adversarial_examples","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/cleverhans-lab/cleverhans","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/daisy611/fusion","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/elites2k19/prism-attack","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/formal-verification-research/NJSMA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/iirishikaii/cleverhans","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/inhopark94/ihpark","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/jbinas/fortified-networks","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/johnsonkee/graduate_design","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/openai/cleverhans","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/probabilistic-jsma/probabilistic-jsma","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/probabilistic-jsmas/probabilistic-jsmas","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/shijiel2/cleverhans","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/tensorflow/cleverhans","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/yaq007/cleverhans","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"practical-black-box-attacks-against-machine","repo_url":"https://github.com/MindCode-4/code-11/tree/main/Black-Box-Attacks-against","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1602.02697","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1602.02697"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tensorflow/cleverhans","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/johnsonkee/graduate_design","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yaq007/cleverhans","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shijiel2/cleverhans","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/probabilistic-jsma/probabilistic-jsma","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/AngusG/cleverhans-attacking-bnns","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/HowToMakeABomb101/Hot2MakeAB0mbSite","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/openai/cleverhans","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindCode-4/code-11/tree/main/Black-Box-Attacks-against","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/probabilistic-jsmas/probabilistic-jsmas","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jbinas/fortified-networks","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/iirishikaii/cleverhans","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/formal-verification-research/NJSMA","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/elites2k19/prism-attack","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/adrian-botta/understanding_adversarial_examples","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cleverhans-lab/cleverhans","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/inhopark94/ihpark","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/daisy611/fusion","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1,"ran_honours":3,"unverified":7},"by_repo_kind":{"listed":{"samples":10,"ran":3,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"50b95028f0c76d76","entry":"arg_type","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"50b95028f0c76d76"}},{"code_sha256_prefix":"b265d88671923e2c","entry":"norm_l0","repo":"probabilistic-jsmas/probabilistic-jsmas","repo_kind":"listed","path":"start_stats.py","file_url":"https://github.com/probabilistic-jsmas/probabilistic-jsmas/blob/HEAD/start_stats.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b265d88671923e2c"}},{"code_sha256_prefix":"3488e2e71e94a1a4","entry":"norm_l1","repo":"probabilistic-jsmas/probabilistic-jsmas","repo_kind":"listed","path":"start_stats.py","file_url":"https://github.com/probabilistic-jsmas/probabilistic-jsmas/blob/HEAD/start_stats.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3488e2e71e94a1a4"}},{"code_sha256_prefix":"824708a335504c38","entry":"norm_l2","repo":"probabilistic-jsmas/probabilistic-jsmas","repo_kind":"listed","path":"start_stats.py","file_url":"https://github.com/probabilistic-jsmas/probabilistic-jsmas/blob/HEAD/start_stats.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"824708a335504c38"}},{"code_sha256_prefix":"9b208439f9e823d7","entry":"compute_rec_err","repo":"jbinas/fortified-networks","repo_kind":"listed","path":"fortnet_dae/utils_tf.py","file_url":"https://github.com/jbinas/fortified-networks/blob/HEAD/fortnet_dae/utils_tf.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b208439f9e823d7"}},{"code_sha256_prefix":"2250a446cd9d9750","entry":"dim2int","repo":"jbinas/fortified-networks","repo_kind":"listed","path":"fortnet_dae/resnet_tf.py","file_url":"https://github.com/jbinas/fortified-networks/blob/HEAD/fortnet_dae/resnet_tf.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2250a446cd9d9750"}},{"code_sha256_prefix":"7802d4d8bfea87e4","entry":"load_mnist","repo":"jbinas/fortified-networks","repo_kind":"listed","path":"datasets/utils_fashion_mnist.py","file_url":"https://github.com/jbinas/fortified-networks/blob/HEAD/datasets/utils_fashion_mnist.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7802d4d8bfea87e4"}},{"code_sha256_prefix":"8f0d60173f7c6db6","entry":"make_resnet_model","repo":"jbinas/fortified-networks","repo_kind":"listed","path":"fortnet_dae/models_tf.py","file_url":"https://github.com/jbinas/fortified-networks/blob/HEAD/fortnet_dae/models_tf.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8f0d60173f7c6db6"}},{"code_sha256_prefix":"befd82aa2271735d","entry":"model_loss","repo":"jbinas/fortified-networks","repo_kind":"listed","path":"fortnet_dae/utils_tf.py","file_url":"https://github.com/jbinas/fortified-networks/blob/HEAD/fortnet_dae/utils_tf.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"befd82aa2271735d"}},{"code_sha256_prefix":"6d943e78fa13a160","entry":"preprocess_image","repo":"jbinas/fortified-networks","repo_kind":"listed","path":"datasets/utils_cifar10.py","file_url":"https://github.com/jbinas/fortified-networks/blob/HEAD/datasets/utils_cifar10.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6d943e78fa13a160"}},{"code_sha256_prefix":"3bb1443aebde61bb","entry":"weight_rec_err","repo":"jbinas/fortified-networks","repo_kind":"listed","path":"run_blackbox.py","file_url":"https://github.com/jbinas/fortified-networks/blob/HEAD/run_blackbox.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3bb1443aebde61bb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}