{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/spectrally-normalized-margin-bounds-for","title":"Spectrally-normalized margin bounds for neural networks","arxiv_id":"1706.08498","date":"2017-06-26","proceeding":"NeurIPS 2017 12","authors":["Peter Bartlett","Dylan J. Foster","Matus Telgarsky"],"abstract":"This paper presents a margin-based multiclass generalization bound for neural\nnetworks that scales with their margin-normalized \"spectral complexity\": their\nLipschitz constant, meaning the product of the spectral norms of the weight\nmatrices, times a certain correction factor. This bound is empirically\ninvestigated for a standard AlexNet network trained with SGD on the mnist and\ncifar10 datasets, with both original and random labels; the bound, the\nLipschitz constants, and the excess risks are all in direct correlation,\nsuggesting both that SGD selects predictors whose complexity scales with the\ndifficulty of the learning task, and secondly that the presented bound is\nsensitive to this complexity.","url_abs":"http://arxiv.org/abs/1706.08498v2","url_pdf":"http://arxiv.org/pdf/1706.08498v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"spectrally-normalized-margin-bounds-for","repo_url":"https://github.com/mostafaelaraby/generalization-gap-features-tensorflow","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"grouped-convolution","method_name":"Grouped Convolution"},{"method_slug":"local-response-normalization","method_name":"Local Response Normalization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"sgd","method_name":"SGD"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1706.08498","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}