Papers › Exploring Dual Model Knowledge Distillation for Anomaly Detection

Exploring Dual Model Knowledge Distillation for Anomaly Detection

27 Jun 2023Preprint 2023 6archive 2025-07-28

Thomine Simon, Snoussi Hichem

Unsupervised anomaly detection holds significant importance in large-scale industrial manufacturing. Recent methods have capitalized on the benefits of utilizing a classifier pretrained on natural images to extract representative features from specific layers. These extracted features are subsequently processed using various techniques. Notably, memory bank-based methods have demonstrated exceptional accuracy; however, they often incur a trade-off in terms of latency. This latency trade-off poses a challenge in real-time industrial applications where prompt anomaly detection and response are crucial. Indeed, alternative approaches such as knowledge distillation and normalized flow have demonstrated promising performance in unsupervised anomaly detection while maintaining low latency. In this paper, we aim to revisit the concept of knowledge distillation in the context of unsupervised anomaly detection, emphasizing the significance of feature selection. By employing distinctive features and leveraging different models, we intend to highlight the importance of carefully selecting and utilizing relevant features specifically tailored for the task of anomaly detection. This article introduces a novel approach based on dual model knowledge distillation for anomaly detection. The proposed method leverages both deep and shallow layers to incorporate various types of semantic information.

Paper PDF

Code

No code repository is listed for this paper in the archive or in Syntology's graph.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Anomaly Detection Knowledge Distillation Unsupervised Anomaly Detection feature selection model

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Anomaly Detection	MVTEC AD textures	DualModel	Detection AUROC	99.94	#1 of 4	Archive leaderboard	report
Anomaly Detection	MVTec AD	DualModel	Detection AUROC	96.2	#82 of 148	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Knowledge Distillation

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections