{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/concentrated-differentially-private-gradient","title":"Concentrated Differentially Private Gradient Descent with Adaptive per-Iteration Privacy Budget","arxiv_id":"1808.09501","date":"2018-08-28","proceeding":null,"authors":["Jaewoo Lee","Daniel Kifer"],"abstract":"Iterative algorithms, like gradient descent, are common tools for solving a\nvariety of problems, such as model fitting. For this reason, there is interest\nin creating differentially private versions of them. However, their conversion\nto differentially private algorithms is often naive. For instance, a fixed\nnumber of iterations are chosen, the privacy budget is split evenly among them,\nand at each iteration, parameters are updated with a noisy gradient. In this\npaper, we show that gradient-based algorithms can be improved by a more careful\nallocation of privacy budget per iteration. Intuitively, at the beginning of\nthe optimization, gradients are expected to be large, so that they do not need\nto be measured as accurately. However, as the parameters approach their optimal\nvalues, the gradients decrease and hence need to be measured more accurately.\nWe add a basic line-search capability that helps the algorithm decide when more\naccurate gradient measurements are necessary. Our gradient descent algorithm\nworks with the recently introduced zCDP version of differential privacy. It\noutperforms prior algorithms for model fitting and is competitive with the\nstate-of-the-art for $(\\epsilon,\\delta)$-differential privacy, a strictly\nweaker definition than zCDP.","url_abs":"http://arxiv.org/abs/1808.09501v1","url_pdf":"http://arxiv.org/pdf/1808.09501v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"concentrated-differentially-private-gradient","repo_url":"https://github.com/ppmlguy/DP-AGD","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1808.09501","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}