Papers › Data Upcycling Knowledge Distillation for Image Super-Resolution

Data Upcycling Knowledge Distillation for Image Super-Resolution

25 Sep 2023arXiv:2309.14162archive 2025-07-28

Yun Zhang, Wei Li, Simiao Li, Hanting Chen, Zhijun Tu, Wenjia Wang, BingYi Jing, Shaohui Lin, Jie Hu

Knowledge distillation (KD) compresses deep neural networks by transferring task-related knowledge from cumbersome pre-trained teacher models to compact student models. However, current KD methods for super-resolution (SR) networks overlook the nature of SR task that the outputs of the teacher model are noisy approximations to the ground-truth distribution of high-quality images (GT), which shades the teacher model's knowledge to result in limited KD effects. To utilize the teacher model beyond the GT upper-bound, we present the Data Upcycling Knowledge Distillation (DUKD), to transfer the teacher model's knowledge to the student model through the upcycled in-domain data derived from training data. Besides, we impose label consistency regularization to KD for SR by the paired invertible augmentations to improve the student model's performance and robustness. Comprehensive experiments demonstrate that the DUKD method significantly outperforms previous arts on several SR tasks.

Paper PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

yun224/dukd officialmentioned in papermentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Image Super-Resolution Knowledge Distillation Quantization Super-Resolution

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Image Super-Resolution	Set14 - 4x upscaling	DUKD	PSNR	28.80	#50 of 104	Archive leaderboard	report
Image Super-Resolution	Set14 - 4x upscaling	DUKD	SSIM	0.7866	#50 of 104	Archive leaderboard	report
Image Super-Resolution	Urban100 - 4x upscaling	RCAN (DUKD)	PSNR	26.62	#32 of 65	Archive leaderboard	report
Image Super-Resolution	Urban100 - 4x upscaling	RCAN (DUKD)	SSIM	0.802	#32 of 65	Archive leaderboard	report
Image Super-Resolution	Urban100 - 4x upscaling	EDSR (DUKD)	PSNR	26.45	#37 of 65	Archive leaderboard	report
Image Super-Resolution	Urban100 - 4x upscaling	EDSR (DUKD)	SSIM	0.7963	#37 of 65	Archive leaderboard	report
Image Super-Resolution	Urban100 - 4x upscaling	SwinIR (DUKD)	PSNR	26.43	#38 of 65	Archive leaderboard	report
Image Super-Resolution	Urban100 - 4x upscaling	SwinIR (DUKD)	SSIM	0.7972	#38 of 65	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Absolute Position Encodings Adam Attention BPE Dense Connections Dropout Knowledge Distillation Label Smoothing Layer Normalization Linear Layer Multi-Head Attention Position-Wise Feed-Forward Layer Residual Connection Softmax Transformer

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections