{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/exploring-dual-model-knowledge-distillation","title":"Exploring Dual Model Knowledge Distillation for Anomaly Detection","arxiv_id":null,"date":"2023-06-27","proceeding":"Preprint 2023 6","authors":["Thomine Simon","Snoussi Hichem"],"abstract":"Unsupervised anomaly detection holds significant importance in large-scale industrial manufacturing. Recent methods have capitalized on the benefits of utilizing\r\na classifier pretrained on natural images to extract representative features from\r\nspecific layers. These extracted features are subsequently processed using various\r\ntechniques. Notably, memory bank-based methods have demonstrated exceptional accuracy; however, they often incur a trade-off in terms of latency.\r\nThis latency trade-off poses a challenge in real-time industrial applications\r\nwhere prompt anomaly detection and response are crucial. Indeed, alternative\r\napproaches such as knowledge distillation and normalized flow have demonstrated\r\npromising performance in unsupervised anomaly detection while maintaining low\r\nlatency. In this paper, we aim to revisit the concept of knowledge distillation\r\nin the context of unsupervised anomaly detection, emphasizing the significance\r\nof feature selection. By employing distinctive features and leveraging different\r\nmodels, we intend to highlight the importance of carefully selecting and utilizing relevant features specifically tailored for the task of anomaly detection. This\r\narticle introduces a novel approach based on dual model knowledge distillation\r\nfor anomaly detection. The proposed method leverages both deep and shallow\r\nlayers to incorporate various types of semantic information.","url_abs":"https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4493018","url_pdf":"https://deliverypdf.ssrn.com/delivery.php?ID=538097071083081007079106111070126005010087014021019030101042034001033016061118121089080088080005011077120091089078122092093023077093071001093031051117071028109107100027070067095007111118093007040004021110064012012098017089065101080122082097115020120067000019102078083105069026064111079115104&EXT=pdf&INDEX=TRUE","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"anomaly-detection","task_name":"Anomaly Detection"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"unsupervised-anomaly-detection","task_name":"Unsupervised Anomaly Detection"},{"task_slug":"feature-selection","task_name":"feature selection"},{"task_slug":"model","task_name":"model"}],"methods":[{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/anomaly-detection-on-mvtec-ad-textures","task":"Anomaly Detection","dataset":"MVTEC AD textures","model":"DualModel","rank_in_archive_order":1,"of":4,"metrics":{"Detection AUROC":"99.94"},"uses_additional_data":false},{"leaderboard":"/sota/anomaly-detection-on-mvtec-ad","task":"Anomaly Detection","dataset":"MVTec AD","model":"DualModel","rank_in_archive_order":82,"of":148,"metrics":{"Detection AUROC":"96.2"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}