{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ksconf-a-light-weight-test-if-a-convnet","title":"KS(conf ): A Light-Weight Test if a ConvNet Operates Outside of Its Specifications","arxiv_id":"1804.04171","date":"2018-04-11","proceeding":null,"authors":["Rémy Sun","Christoph H. Lampert"],"abstract":"Computer vision systems for automatic image categorization have become\naccurate and reliable enough that they can run continuously for days or even\nyears as components of real-world commercial applications. A major open problem\nin this context, however, is quality control. Good classification performance\ncan only be expected if systems run under the specific conditions, in\nparticular data distributions, that they were trained for. Surprisingly, none\nof the currently used deep network architectures has a built-in functionality\nthat could detect if a network operates on data from a distribution that it was\nnot trained for and potentially trigger a warning to the human users. In this\nwork, we describe KS(conf), a procedure for detecting such outside of the\nspecifications operation. Building on statistical insights, its main step is\nthe applications of a classical Kolmogorov-Smirnov test to the distribution of\npredicted confidence values. We show by extensive experiments using ImageNet,\nAwA2 and DAVIS data on a variety of ConvNets architectures that KS(conf)\nreliably detects out-of-specs situations. It furthermore has a number of\nproperties that make it an excellent candidate for practical deployment: it is\neasy to implement, adds almost no overhead to the system, works with all\nnetworks, including pretrained ones, and requires no a priori knowledge about\nhow the data distribution could change.","url_abs":"http://arxiv.org/abs/1804.04171v1","url_pdf":"http://arxiv.org/pdf/1804.04171v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ksconf-a-light-weight-test-if-a-convnet","repo_url":"https://github.com/ISTAustria-CVML/KSconf","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"image-categorization","task_name":"Image Categorization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}