{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/differential-privacy-has-disparate-impact-on","title":"Differential Privacy Has Disparate Impact on Model Accuracy","arxiv_id":"1905.12101","date":"2019-05-28","proceeding":"NeurIPS 2019 12","authors":["Eugene Bagdasaryan","Vitaly Shmatikov"],"abstract":"Differential privacy (DP) is a popular mechanism for training machine learning models with bounded leakage about the presence of specific points in the training data. The cost of differential privacy is a reduction in the model's accuracy. We demonstrate that in the neural networks trained using differentially private stochastic gradient descent (DP-SGD), this cost is not borne equally: accuracy of DP models drops much more for the underrepresented classes and subgroups. For example, a gender classification model trained using DP-SGD exhibits much lower accuracy for black faces than for white faces. Critically, this gap is bigger in the DP model than in the non-DP model, i.e., if the original model is unfair, the unfairness becomes worse once DP is applied. We demonstrate this effect for a variety of tasks and models, including sentiment analysis of text and image classification. We then explain why DP training mechanisms such as gradient clipping and noise addition have disproportionate effect on the underrepresented and more complex subgroups, resulting in a disparate reduction of model accuracy.","url_abs":"https://arxiv.org/abs/1905.12101v2","url_pdf":"https://arxiv.org/pdf/1905.12101v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"differential-privacy-has-disparate-impact-on","repo_url":"https://github.com/ebagdasa/differential-privacy-vs-fairness","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"gender-classification","task_name":"Gender Classification"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":"image-classification","task_name":"image-classification"},{"task_slug":"model","task_name":"model"}],"methods":[{"method_slug":"gradient-clipping","method_name":"Gradient Clipping"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1905.12101","atlas_url":"https://app.syntology.ai/?focus=1905.12101","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1905.12101"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ebagdasa/differential-privacy-vs-fairness","reach":{"status":"ok"}}],"summary":{"ran_honours":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"b9fda299a49d212a","entry":"compute_norm","repo":"ebagdasa/differential-privacy-vs-fairness","repo_kind":"official","path":"playing.py","file_url":"https://github.com/ebagdasa/differential-privacy-vs-fairness/blob/HEAD/playing.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b9fda299a49d212a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}