{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ole-orthogonal-low-rank-embedding-a-plug-and","title":"OLÉ: Orthogonal Low-rank Embedding, A Plug and Play Geometric Loss for Deep Learning","arxiv_id":"1712.01727","date":"2017-12-05","proceeding":null,"authors":["José Lezama","Qiang Qiu","Pablo Musé","Guillermo Sapiro"],"abstract":"Deep neural networks trained using a softmax layer at the top and the\ncross-entropy loss are ubiquitous tools for image classification. Yet, this\ndoes not naturally enforce intra-class similarity nor inter-class margin of the\nlearned deep representations. To simultaneously achieve these two goals,\ndifferent solutions have been proposed in the literature, such as the pairwise\nor triplet losses. However, such solutions carry the extra task of selecting\npairs or triplets, and the extra computational burden of computing and learning\nfor many combinations of them. In this paper, we propose a plug-and-play loss\nterm for deep networks that explicitly reduces intra-class variance and\nenforces inter-class margin simultaneously, in a simple and elegant geometric\nmanner. For each class, the deep features are collapsed into a learned linear\nsubspace, or union of them, and inter-class subspaces are pushed to be as\northogonal as possible. Our proposed Orthogonal Low-rank Embedding (OL\\'E) does\nnot require carefully crafting pairs or triplets of samples for training, and\nworks standalone as a classification loss, being the first reported deep metric\nlearning framework of its kind. Because of the improved margin between features\nof different classes, the resulting deep networks generalize better, are more\ndiscriminative, and more robust. We demonstrate improved classification\nperformance in general object recognition, plugging the proposed loss term into\nexisting off-the-shelf architectures. In particular, we show the advantage of\nthe proposed loss in the small data/model scenario, and we significantly\nadvance the state-of-the-art on the Stanford STL-10 benchmark.","url_abs":"http://arxiv.org/abs/1712.01727v1","url_pdf":"http://arxiv.org/pdf/1712.01727v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ole-orthogonal-low-rank-embedding-a-plug-and","repo_url":"https://github.com/jlezama/OrthogonalLowrankEmbedding","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"metric-learning","task_name":"Metric Learning"},{"task_slug":"object-recognition","task_name":"Object Recognition"},{"task_slug":"small-data","task_name":"Small Data Image Classification"},{"task_slug":null,"task_name":"Triplet"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1712.01727","atlas_url":"https://app.syntology.ai/?focus=1712.01727","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}