{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/privacy-for-free-in-the-over-parameterized","title":"Privacy for Free in the Overparameterized Regime","arxiv_id":"2410.14787","date":"2024-10-18","proceeding":null,"authors":["Simone Bombari","Marco Mondelli"],"abstract":"Differentially private gradient descent (DP-GD) is a popular algorithm to train deep learning models with provable guarantees on the privacy of the training data. In the last decade, the problem of understanding its performance cost with respect to standard GD has received remarkable attention from the research community, which formally derived upper bounds on the excess population risk $R_{P}$ in different learning settings. However, existing bounds typically degrade with over-parameterization, i.e., as the number of parameters $p$ gets larger than the number of training samples $n$ -- a regime which is ubiquitous in current deep-learning practice. As a result, the lack of theoretical insights leaves practitioners without clear guidance, leading some to reduce the effective number of trainable parameters to improve performance, while others use larger models to achieve better results through scale. In this work, we show that in the popular random features model with quadratic loss, for any sufficiently large $p$, privacy can be obtained for free, i.e., $\\left|R_{P} \\right| = o(1)$, not only when the privacy parameter $\\varepsilon$ has constant order, but also in the strongly private setting $\\varepsilon = o(1)$. This challenges the common wisdom that over-parameterization inherently hinders performance in private learning.","url_abs":"https://arxiv.org/abs/2410.14787v2","url_pdf":"https://arxiv.org/pdf/2410.14787v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"privacy-for-free-in-the-over-parameterized","repo_url":"https://github.com/simone-bombari/privacy-for-free","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"jax","reach":null}],"tasks":[],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.14787","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.14787"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/simone-bombari/privacy-for-free","reach":null}],"summary":{"ran_draft_wrong":1,"ran_fixture":1,"ran_honours":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"724b2f122c38aaa1","entry":"initialize_params","repo":"simone-bombari/privacy-for-free","repo_kind":"official","path":"utils_NN_JAX.py","file_url":"https://github.com/simone-bombari/privacy-for-free/blob/HEAD/utils_NN_JAX.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"724b2f122c38aaa1"}},{"code_sha256_prefix":"e978ea7779d34d73","entry":"predict","repo":"simone-bombari/privacy-for-free","repo_kind":"official","path":"utils_NN_JAX.py","file_url":"https://github.com/simone-bombari/privacy-for-free/blob/HEAD/utils_NN_JAX.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e978ea7779d34d73"}},{"code_sha256_prefix":"85adc5760af572a0","entry":"tanh","repo":"simone-bombari/privacy-for-free","repo_kind":"official","path":"main_RF.py","file_url":"https://github.com/simone-bombari/privacy-for-free/blob/HEAD/main_RF.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"85adc5760af572a0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}