{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dp-sgd-without-clipping-the-lipschitz-neural","title":"DP-SGD Without Clipping: The Lipschitz Neural Network Way","arxiv_id":"2305.16202","date":"2023-05-25","proceeding":null,"authors":["Louis Bethune","Thomas Massena","Thibaut Boissin","Yannick Prudent","Corentin Friedrich","Franck Mamalet","Aurelien Bellet","Mathieu Serrurier","David Vigouroux"],"abstract":"State-of-the-art approaches for training Differentially Private (DP) Deep Neural Networks (DNN) face difficulties to estimate tight bounds on the sensitivity of the network's layers, and instead rely on a process of per-sample gradient clipping. This clipping process not only biases the direction of gradients but also proves costly both in memory consumption and in computation. To provide sensitivity bounds and bypass the drawbacks of the clipping process, we propose to rely on Lipschitz constrained networks. Our theoretical analysis reveals an unexplored link between the Lipschitz constant with respect to their input and the one with respect to their parameters. By bounding the Lipschitz constant of each layer with respect to its parameters, we prove that we can train these networks with privacy guarantees. Our analysis not only allows the computation of the aforementioned sensitivities at scale, but also provides guidance on how to maximize the gradient-to-noise ratio for fixed privacy guarantees. The code has been released as a Python package available at https://github.com/Algue-Rythme/lip-dp","url_abs":"https://arxiv.org/abs/2305.16202v2","url_pdf":"https://arxiv.org/pdf/2305.16202v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dp-sgd-without-clipping-the-lipschitz-neural","repo_url":"https://github.com/algue-rythme/lip-dp","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"sensitivity","task_name":"Sensitivity"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.16202","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.16202"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Algue-Rythme/lip-dp","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/algue-rythme/lip-dp","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":1,"unverified":14},"by_repo_kind":{"official":{"samples":15,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7dbb68c805496854","entry":"above_treshold","repo":"algue-rythme/lip-dp","repo_kind":"official","path":"deel/lipdp/model.py","file_url":"https://github.com/algue-rythme/lip-dp/blob/HEAD/deel/lipdp/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7dbb68c805496854"}},{"code_sha256_prefix":"3c371e3edb645884","entry":"DP_GNP_Factory","repo":"Algue-Rythme/lip-dp","repo_kind":"official","path":"deel/lipdp/layers.py","file_url":"https://github.com/Algue-Rythme/lip-dp/blob/HEAD/deel/lipdp/layers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3c371e3edb645884"}},{"code_sha256_prefix":"42fa1baaddd7b892","entry":"accuracy","repo":"Algue-Rythme/lip-dp","repo_kind":"official","path":"experiments/paper_plots/opacus_cifar10.py","file_url":"https://github.com/Algue-Rythme/lip-dp/blob/HEAD/experiments/paper_plots/opacus_cifar10.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"42fa1baaddd7b892"}},{"code_sha256_prefix":"70bc35deeaf40447","entry":"bound_clip_value","repo":"Algue-Rythme/lip-dp","repo_kind":"official","path":"deel/lipdp/pipeline.py","file_url":"https://github.com/Algue-Rythme/lip-dp/blob/HEAD/deel/lipdp/pipeline.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"70bc35deeaf40447"}},{"code_sha256_prefix":"a4d31069ca18e5ba","entry":"check_layer_gradient_norm","repo":"Algue-Rythme/lip-dp","repo_kind":"official","path":"deel/lipdp/sensitivity.py","file_url":"https://github.com/Algue-Rythme/lip-dp/blob/HEAD/deel/lipdp/sensitivity.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a4d31069ca18e5ba"}},{"code_sha256_prefix":"2fc611adb0c47358","entry":"clip_gradient","repo":"Algue-Rythme/lip-dp","repo_kind":"official","path":"deel/lipdp/layers.py","file_url":"https://github.com/Algue-Rythme/lip-dp/blob/HEAD/deel/lipdp/layers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2fc611adb0c47358"}},{"code_sha256_prefix":"bb45c1b9dadaffa3","entry":"clipsum","repo":"Algue-Rythme/lip-dp","repo_kind":"official","path":"deel/lipdp/model.py","file_url":"https://github.com/Algue-Rythme/lip-dp/blob/HEAD/deel/lipdp/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bb45c1b9dadaffa3"}},{"code_sha256_prefix":"19eda6e38504d4cf","entry":"clipsum","repo":"Algue-Rythme/lip-dp","repo_kind":"official","path":"deel/lipdp/dynamic.py","file_url":"https://github.com/Algue-Rythme/lip-dp/blob/HEAD/deel/lipdp/dynamic.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"19eda6e38504d4cf"}},{"code_sha256_prefix":"697898c4e39ba9f5","entry":"diff_query","repo":"Algue-Rythme/lip-dp","repo_kind":"official","path":"deel/lipdp/model.py","file_url":"https://github.com/Algue-Rythme/lip-dp/blob/HEAD/deel/lipdp/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"697898c4e39ba9f5"}},{"code_sha256_prefix":"bc040ca918d02a95","entry":"diff_query","repo":"Algue-Rythme/lip-dp","repo_kind":"official","path":"deel/lipdp/dynamic.py","file_url":"https://github.com/Algue-Rythme/lip-dp/blob/HEAD/deel/lipdp/dynamic.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bc040ca918d02a95"}},{"code_sha256_prefix":"b9674ef0be894aaa","entry":"get_colorspace_function","repo":"Algue-Rythme/lip-dp","repo_kind":"official","path":"deel/lipdp/pipeline.py","file_url":"https://github.com/Algue-Rythme/lip-dp/blob/HEAD/deel/lipdp/pipeline.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b9674ef0be894aaa"}},{"code_sha256_prefix":"ebcc53c313eecd25","entry":"laplace_above_treshold","repo":"Algue-Rythme/lip-dp","repo_kind":"official","path":"deel/lipdp/dynamic.py","file_url":"https://github.com/Algue-Rythme/lip-dp/blob/HEAD/deel/lipdp/dynamic.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ebcc53c313eecd25"}},{"code_sha256_prefix":"12b077431b025fd0","entry":"make_residuals","repo":"Algue-Rythme/lip-dp","repo_kind":"official","path":"deel/lipdp/layers.py","file_url":"https://github.com/Algue-Rythme/lip-dp/blob/HEAD/deel/lipdp/layers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"12b077431b025fd0"}},{"code_sha256_prefix":"bc270f2e54ccfbdc","entry":"test","repo":"Algue-Rythme/lip-dp","repo_kind":"official","path":"experiments/paper_plots/opacus_cifar10.py","file_url":"https://github.com/Algue-Rythme/lip-dp/blob/HEAD/experiments/paper_plots/opacus_cifar10.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bc270f2e54ccfbdc"}},{"code_sha256_prefix":"f7cf089e90afe382","entry":"test","repo":"Algue-Rythme/lip-dp","repo_kind":"official","path":"experiments/paper_plots/opacus_tabular.py","file_url":"https://github.com/Algue-Rythme/lip-dp/blob/HEAD/experiments/paper_plots/opacus_tabular.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f7cf089e90afe382"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}