{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/large-margin-deep-networks-for-classification","title":"Large Margin Deep Networks for Classification","arxiv_id":"1803.05598","date":"2018-03-15","proceeding":"NeurIPS 2018 12","authors":["Gamaleldin F. Elsayed","Dilip Krishnan","Hossein Mobahi","Kevin Regan","Samy Bengio"],"abstract":"We present a formulation of deep learning that aims at producing a large\nmargin classifier. The notion of margin, minimum distance to a decision\nboundary, has served as the foundation of several theoretically profound and\nempirically successful results for both classification and regression tasks.\nHowever, most large margin algorithms are applicable only to shallow models\nwith a preset feature representation; and conventional margin methods for\nneural networks only enforce margin at the output layer. Such methods are\ntherefore not well suited for deep networks.\n  In this work, we propose a novel loss function to impose a margin on any\nchosen set of layers of a deep network (including input and hidden layers). Our\nformulation allows choosing any norm on the metric measuring the margin. We\ndemonstrate that the decision boundary obtained by our loss has nice properties\ncompared to standard classification loss functions. Specifically, we show\nimproved empirical results on the MNIST, CIFAR-10 and ImageNet datasets on\nmultiple tasks: generalization from small training sets, corrupted labels, and\nrobustness against adversarial perturbations. The resulting loss is general and\ncomplementary to existing data augmentation (such as random/adversarial input\ntransform) and regularization techniques (such as weight decay, dropout, and\nbatch norm).","url_abs":"http://arxiv.org/abs/1803.05598v2","url_pdf":"http://arxiv.org/pdf/1803.05598v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"large-margin-deep-networks-for-classification","repo_url":"https://github.com/google-research/google-research","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null},{"paper_slug":"large-margin-deep-networks-for-classification","repo_url":"https://github.com/zsef123/Large_Margin_Loss_PyTorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"classification","task_name":"General Classification"}],"methods":[{"method_slug":"affine-coupling","method_name":"Affine Coupling"},{"method_slug":"normalizing-flows","method_name":"Normalizing Flows"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1803.05598","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}