{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-local-elasticity-of-neural-networks","title":"The Local Elasticity of Neural Networks","arxiv_id":"1910.06943","date":"2019-10-15","proceeding":"ICLR 2020 1","authors":["Hangfeng He","Weijie J. Su"],"abstract":"This paper presents a phenomenon in neural networks that we refer to as \\textit{local elasticity}. Roughly speaking, a classifier is said to be locally elastic if its prediction at a feature vector $\\bx'$ is \\textit{not} significantly perturbed, after the classifier is updated via stochastic gradient descent at a (labeled) feature vector $\\bx$ that is \\textit{dissimilar} to $\\bx'$ in a certain sense. This phenomenon is shown to persist for neural networks with nonlinear activation functions through extensive simulations on real-life and synthetic datasets, whereas this is not observed in linear classifiers. In addition, we offer a geometric interpretation of local elasticity using the neural tangent kernel \\citep{jacot2018neural}. Building on top of local elasticity, we obtain pairwise similarity measures between feature vectors, which can be used for clustering in conjunction with $K$-means. The effectiveness of the clustering algorithm on the MNIST and CIFAR-10 datasets in turn corroborates the hypothesis of local elasticity of neural networks on real-life data. Finally, we discuss some implications of local elasticity to shed light on several intriguing aspects of deep neural networks.","url_abs":"https://arxiv.org/abs/1910.06943v2","url_pdf":"https://arxiv.org/pdf/1910.06943v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-local-elasticity-of-neural-networks","repo_url":"https://github.com/HornHehhf/LocalElasticity","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"clustering","task_name":"Clustering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1910.06943","atlas_url":"https://app.syntology.ai/?focus=1910.06943","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}