{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/how-to-make-the-gradients-small-privately","title":"How to Make the Gradients Small Privately: Improved Rates for Differentially Private Non-Convex Optimization","arxiv_id":"2402.11173","date":"2024-02-17","proceeding":null,"authors":["Andrew Lowy","Jonathan Ullman","Stephen J. Wright"],"abstract":"We provide a simple and flexible framework for designing differentially private algorithms to find approximate stationary points of non-convex loss functions. Our framework is based on using a private approximate risk minimizer to \"warm start\" another private algorithm for finding stationary points. We use this framework to obtain improved, and sometimes optimal, rates for several classes of non-convex loss functions. First, we obtain improved rates for finding stationary points of smooth non-convex empirical loss functions. Second, we specialize to quasar-convex functions, which generalize star-convex functions and arise in learning dynamical systems and training some neural nets. We achieve the optimal rate for this class. Third, we give an optimal algorithm for finding stationary points of functions satisfying the Kurdyka-Lojasiewicz (KL) condition. For example, over-parameterized neural networks often satisfy this condition. Fourth, we provide new state-of-the-art rates for stationary points of non-convex population loss functions. Fifth, we obtain improved rates for non-convex generalized linear models. A modification of our algorithm achieves nearly the same rates for second-order stationary points of functions with Lipschitz Hessian, improving over the previous state-of-the-art for each of the above problems.","url_abs":"https://arxiv.org/abs/2402.11173v2","url_pdf":"https://arxiv.org/pdf/2402.11173v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"how-to-make-the-gradients-small-privately","repo_url":"https://github.com/lowya/how-to-make-the-gradients-small-privately","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"CC0-1.0"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2402.11173","atlas_url":"https://app.syntology.ai/?focus=2402.11173","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.11173"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lowya/how-to-make-the-gradients-small-privately","reach":{"status":"ok","spdx":"CC0-1.0"}}],"summary":{"ran":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5d5c9ec5efaf2f23","entry":"grad","repo":"lowya/how-to-make-the-gradients-small-privately","repo_kind":"official","path":"mod_smallgradients_trig_icml24_V3.py","file_url":"https://github.com/lowya/how-to-make-the-gradients-small-privately/blob/HEAD/mod_smallgradients_trig_icml24_V3.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"CC0-1.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5d5c9ec5efaf2f23"}},{"code_sha256_prefix":"d7244e1449bbaf90","entry":"grad_norm","repo":"lowya/how-to-make-the-gradients-small-privately","repo_kind":"official","path":"mod_smallgradients_trig_icml24_V3.py","file_url":"https://github.com/lowya/how-to-make-the-gradients-small-privately/blob/HEAD/mod_smallgradients_trig_icml24_V3.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC0-1.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d7244e1449bbaf90"}},{"code_sha256_prefix":"ac7f91a5ccbb1c83","entry":"sample_indices","repo":"lowya/how-to-make-the-gradients-small-privately","repo_kind":"official","path":"mod_smallgradients_trig_icml24_V3.py","file_url":"https://github.com/lowya/how-to-make-the-gradients-small-privately/blob/HEAD/mod_smallgradients_trig_icml24_V3.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC0-1.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ac7f91a5ccbb1c83"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}