{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/smoothnets-optimizing-cnn-architecture-design","title":"SmoothNets: Optimizing CNN architecture design for differentially private deep learning","arxiv_id":"2205.04095","date":"2022-05-09","proceeding":null,"authors":["Nicolas W. Remerscheid","Alexander Ziller","Daniel Rueckert","Georgios Kaissis"],"abstract":"The arguably most widely employed algorithm to train deep neural networks with Differential Privacy is DPSGD, which requires clipping and noising of per-sample gradients. This introduces a reduction in model utility compared to non-private training. Empirically, it can be observed that this accuracy degradation is strongly dependent on the model architecture. We investigated this phenomenon and, by combining components which exhibit good individual performance, distilled a new model architecture termed SmoothNet, which is characterised by increased robustness to the challenges of DP-SGD training. Experimentally, we benchmark SmoothNet against standard architectures on two benchmark datasets and observe that our architecture outperforms others, reaching an accuracy of 73.5\\% on CIFAR-10 at $\\varepsilon=7.0$ and 69.2\\% at $\\varepsilon=7.0$ on ImageNette, a state-of-the-art result compared to prior architectural modifications for DP.","url_abs":"https://arxiv.org/abs/2205.04095v1","url_pdf":"https://arxiv.org/pdf/2205.04095v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"smoothnets-optimizing-cnn-architecture-design","repo_url":"https://github.com/NiWaRe/DPBenchmark","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-classification-with-dp","task_name":"Image Classification with Differential Privacy"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-cifar-10","task":"Image Classification","dataset":"CIFAR-10","model":"SmoothNetV1","rank_in_archive_order":259,"of":265,"metrics":{"Percentage correct":"73.5"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenette","task":"Image Classification","dataset":"Imagenette","model":"SmoothNetV1","rank_in_archive_order":2,"of":2,"metrics":{"Accuracy":"69.7"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}