{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/190411005","title":"Analytical Moment Regularizer for Gaussian Robust Networks","arxiv_id":"1904.11005","date":"2019-04-24","proceeding":null,"authors":["Modar Alfadly","Adel Bibi","Bernard Ghanem"],"abstract":"Despite the impressive performance of deep neural networks (DNNs) on numerous\nvision tasks, they still exhibit yet-to-understand uncouth behaviours. One\npuzzling behaviour is the subtle sensitive reaction of DNNs to various noise\nattacks. Such a nuisance has strengthened the line of research around\ndeveloping and training noise-robust networks. In this work, we propose a new\ntraining regularizer that aims to minimize the probabilistic expected training\nloss of a DNN subject to a generic Gaussian input. We provide an efficient and\nsimple approach to approximate such a regularizer for arbitrary deep networks.\nThis is done by leveraging the analytic expression of the output mean of a\nshallow neural network; avoiding the need for the memory and computationally\nexpensive data augmentation. We conduct extensive experiments on LeNet and\nAlexNet on various datasets including MNIST, CIFAR10, and CIFAR100\ndemonstrating the effectiveness of our proposed regularizer. In particular, we\nshow that networks that are trained with the proposed regularizer benefit from\na boost in robustness equivalent to performing 3-21 folds of data augmentation.","url_abs":"http://arxiv.org/abs/1904.11005v1","url_pdf":"http://arxiv.org/pdf/1904.11005v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"190411005","repo_url":"https://github.com/ModarTensai/gaussian-regularizer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"lenet","method_name":"LeNet"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}