{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fast-single-class-classification-and-the","title":"Fast Single-Class Classification and the Principle of Logit Separation","arxiv_id":"1705.10246","date":"2017-05-29","proceeding":null,"authors":["Gil Keren","Sivan Sabato","Björn Schuller"],"abstract":"We consider neural network training, in applications in which there are many\npossible classes, but at test-time, the task is a binary classification task of\ndetermining whether the given example belongs to a specific class, where the\nclass of interest can be different each time the classifier is applied. For\ninstance, this is the case for real-time image search. We define the Single\nLogit Classification (SLC) task: training the network so that at test-time, it\nwould be possible to accurately identify whether the example belongs to a given\nclass in a computationally efficient manner, based only on the output logit for\nthis class. We propose a natural principle, the Principle of Logit Separation,\nas a guideline for choosing and designing losses suitable for the SLC. We show\nthat the cross-entropy loss function is not aligned with the Principle of Logit\nSeparation. In contrast, there are known loss functions, as well as novel batch\nloss functions that we propose, which are aligned with this principle. In\ntotal, we study seven loss functions. Our experiments show that indeed in\nalmost all cases, losses that are aligned with the Principle of Logit\nSeparation obtain at least 20% relative accuracy improvement in the SLC task\ncompared to losses that are not aligned with it, and sometimes considerably\nmore. Furthermore, we show that fast SLC does not cause any drop in binary\nclassification accuracy, compared to standard classification in which all\nlogits are computed, and yields a speedup which grows with the number of\nclasses. For instance, we demonstrate a 10x speedup when the number of classes\nis 400,000. Tensorflow code for optimizing the new batch losses is publicly\navailable at https://github.com/cruvadom/Logit Separation.","url_abs":"http://arxiv.org/abs/1705.10246v4","url_pdf":"http://arxiv.org/pdf/1705.10246v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fast-single-class-classification-and-the","repo_url":"https://github.com/cruvadom/Logit_Separation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"fast-single-class-classification-and-the","repo_url":"https://github.com/EIHW/Logit_Separation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"binary-classification","task_name":"Binary Classification"},{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"image-retrieval","task_name":"Image Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}