{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/only-sparsity-based-loss-function-for","title":"Only sparsity based loss function for learning representations","arxiv_id":"1903.02893","date":"2019-03-07","proceeding":null,"authors":["Vivek Bakaraju","Kishore Reddy Konda"],"abstract":"We study the emergence of sparse representations in neural networks. We show\nthat in unsupervised models with regularization, the emergence of sparsity is\nthe result of the input data samples being distributed along highly non-linear\nor discontinuous manifold. We also derive a similar argument for\ndiscriminatively trained networks and present experiments to support this\nhypothesis. Based on our study of sparsity, we introduce a new loss function\nwhich can be used as regularization term for models like autoencoders and MLPs.\nFurther, the same loss function can also be used as a cost function for an\nunsupervised single-layered neural network model for learning efficient\nrepresentations.","url_abs":"http://arxiv.org/abs/1903.02893v1","url_pdf":"http://arxiv.org/pdf/1903.02893v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"only-sparsity-based-loss-function-for","repo_url":"https://github.com/Vivek-B/OVR","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}