{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/do-not-let-privacy-overbill-utility-gradient-1","title":"Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning","arxiv_id":"2102.12677","date":"2021-02-25","proceeding":"ICLR 2021 1","authors":["Da Yu","Huishuai Zhang","Wei Chen","Tie-Yan Liu"],"abstract":"The privacy leakage of the model about the training data can be bounded in the differential privacy mechanism. However, for meaningful privacy parameters, a differentially private model degrades the utility drastically when the model comprises a large number of trainable parameters. In this paper, we propose an algorithm \\emph{Gradient Embedding Perturbation (GEP)} towards training differentially private deep models with decent accuracy. Specifically, in each gradient descent step, GEP first projects individual private gradient into a non-sensitive anchor subspace, producing a low-dimensional gradient embedding and a small-norm residual gradient. Then, GEP perturbs the low-dimensional embedding and the residual gradient separately according to the privacy budget. Such a decomposition permits a small perturbation variance, which greatly helps to break the dimensional barrier of private learning. With GEP, we achieve decent accuracy with reasonable computational cost and modest privacy guarantee for deep models. Especially, with privacy bound $\\epsilon=8$, we achieve $74.9\\%$ test accuracy on CIFAR10 and $95.1\\%$ test accuracy on SVHN, significantly improving over existing results.","url_abs":"https://arxiv.org/abs/2102.12677v3","url_pdf":"https://arxiv.org/pdf/2102.12677v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"do-not-let-privacy-overbill-utility-gradient-1","repo_url":"https://github.com/dayu11/Gradient-Embedding-Perturbation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"do-not-let-privacy-overbill-utility-gradient-1","repo_url":"https://github.com/dayu11/Differentially-Private-Deep-Learning","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2102.12677","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2102.12677"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dayu11/Gradient-Embedding-Perturbation","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dayu11/Differentially-Private-Deep-Learning","reach":null}],"summary":{"unverified":1},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"864390d230441893","entry":"conv3x3","repo":"dayu11/Gradient-Embedding-Perturbation","repo_kind":"official","path":"models/resnet_cifar.py","file_url":"https://github.com/dayu11/Gradient-Embedding-Perturbation/blob/HEAD/models/resnet_cifar.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"864390d230441893"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}