{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/toward-training-at-imagenet-scale-with","title":"Toward Training at ImageNet Scale with Differential Privacy","arxiv_id":"2201.12328","date":"2022-01-28","proceeding":null,"authors":["Alexey Kurakin","Shuang Song","Steve Chien","Roxana Geambasu","Andreas Terzis","Abhradeep Thakurta"],"abstract":"Differential privacy (DP) is the de facto standard for training machine learning (ML) models, including neural networks, while ensuring the privacy of individual examples in the training set. Despite a rich literature on how to train ML models with differential privacy, it remains extremely challenging to train real-life, large neural networks with both reasonable accuracy and privacy. We set out to investigate how to do this, using ImageNet image classification as a poster example of an ML task that is very challenging to resolve accurately with DP right now. This paper shares initial lessons from our effort, in the hope that it will inspire and inform other researchers to explore DP training at scale. We show approaches that help make DP training faster, as well as model types and settings of the training process that tend to work better in the DP setting. Combined, the methods we discuss let us train a Resnet-18 with DP to $47.9\\%$ accuracy and privacy parameters $\\epsilon = 10, \\delta = 10^{-6}$. This is a significant improvement over \"naive\" DP training of ImageNet models, but a far cry from the $75\\%$ accuracy that can be obtained by the same network without privacy. The model we use was pretrained on the Places365 data set as a starting point. We share our code at https://github.com/google-research/dp-imagenet, calling for others to build upon this new baseline to further improve DP at scale.","url_abs":"https://arxiv.org/abs/2201.12328v2","url_pdf":"https://arxiv.org/pdf/2201.12328v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"toward-training-at-imagenet-scale-with","repo_url":"https://github.com/google-research/dp-imagenet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-classification-with-dp","task_name":"Image Classification with Differential Privacy"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-with-dp-on-imagenet","task":"Image Classification with Differential Privacy","dataset":"ImageNet","model":"Resnet-18","rank_in_archive_order":3,"of":4,"metrics":{"Top 1 Accuracy":"6.9"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-with-dp-on-imagenet","task":"Image Classification with Differential Privacy","dataset":"ImageNet","model":"Resnet-50","rank_in_archive_order":4,"of":4,"metrics":{"Top 1 Accuracy":"5"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2201.12328","atlas_url":"https://app.syntology.ai/?focus=2201.12328","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2201.12328"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-research/dp-imagenet","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":6,"unverified":2},"by_repo_kind":{"official":{"samples":8,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bb017e937333f8cb","entry":"convnet","repo":"google-research/dp-imagenet","repo_kind":"official","path":"benchmark/cifar10_opacus.py","file_url":"https://github.com/google-research/dp-imagenet/blob/HEAD/benchmark/cifar10_opacus.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bb017e937333f8cb"}},{"code_sha256_prefix":"394c0749c7ba9f69","entry":"normalize_images","repo":"google-research/dp-imagenet","repo_kind":"official","path":"benchmark/cifar10_objax.py","file_url":"https://github.com/google-research/dp-imagenet/blob/HEAD/benchmark/cifar10_objax.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"394c0749c7ba9f69"}},{"code_sha256_prefix":"a8393da046583d5d","entry":"normalize_images","repo":"google-research/dp-imagenet","repo_kind":"official","path":"benchmark/cifar10_tf.py","file_url":"https://github.com/google-research/dp-imagenet/blob/HEAD/benchmark/cifar10_tf.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a8393da046583d5d"}},{"code_sha256_prefix":"4221b2cc1686f00c","entry":"test","repo":"google-research/dp-imagenet","repo_kind":"official","path":"benchmark/mnist_opacus.py","file_url":"https://github.com/google-research/dp-imagenet/blob/HEAD/benchmark/mnist_opacus.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4221b2cc1686f00c"}},{"code_sha256_prefix":"e856757b66bf0168","entry":"train","repo":"google-research/dp-imagenet","repo_kind":"official","path":"benchmark/cifar10_objax.py","file_url":"https://github.com/google-research/dp-imagenet/blob/HEAD/benchmark/cifar10_objax.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e856757b66bf0168"}},{"code_sha256_prefix":"605992b94d3a3108","entry":"train","repo":"google-research/dp-imagenet","repo_kind":"official","path":"benchmark/mnist_opacus.py","file_url":"https://github.com/google-research/dp-imagenet/blob/HEAD/benchmark/mnist_opacus.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"605992b94d3a3108"}},{"code_sha256_prefix":"42fa1baaddd7b892","entry":"accuracy","repo":"google-research/dp-imagenet","repo_kind":"official","path":"benchmark/cifar10_opacus.py","file_url":"https://github.com/google-research/dp-imagenet/blob/HEAD/benchmark/cifar10_opacus.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"42fa1baaddd7b892"}},{"code_sha256_prefix":"ff2ef7a99e4a92ac","entry":"train","repo":"google-research/dp-imagenet","repo_kind":"official","path":"benchmark/cifar10_opacus.py","file_url":"https://github.com/google-research/dp-imagenet/blob/HEAD/benchmark/cifar10_opacus.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ff2ef7a99e4a92ac"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}