Papers › Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel...

Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network

16 Sep 2016CVPR 2016 6arXiv:1609.05158archive 2025-07-28

Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P. Aitken, Rob Bishop, Daniel Rueckert, Zehan Wang

Recently, several models based on deep neural networks have achieved great success in terms of both reconstruction accuracy and computational performance for single image super-resolution. In these methods, the low resolution (LR) input image is upscaled to the high resolution (HR) space using a single filter, commonly bicubic interpolation, before reconstruction. This means that the super-resolution (SR) operation is performed in HR space. We demonstrate that this is sub-optimal and adds computational complexity. In this paper, we present the first convolutional neural network (CNN) capable of real-time SR of 1080p videos on a single K2 GPU. To achieve this, we propose a novel CNN architecture where the feature maps are extracted in the LR space. In addition, we introduce an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output. By doing so, we effectively replace the handcrafted bicubic filter in the SR pipeline with more complex upscaling filters specifically trained for each feature map, whilst also reducing the computational complexity of the overall SR operation. We evaluate the proposed approach using images and videos from publicly available datasets and show that it performs significantly better (+0.15dB on Images and +0.39dB on Videos) and is an order of magnitude faster than previous CNN-based methods.

PaperPDFConference PDFCodeCode Syntology ran

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, Syntology's MCP tool lists every function and class Syntology harvested from this paper and whether it ran (how to connect): get_harvested_code_for_paper(arxiv_id="1609.05158")

Code

Syntology Ran 4 of 20 code samples harvested from 8 repositories linked to this paper; 16 have no recorded run. Of those that ran: 2 ran · our draft was wrong; 2 ran · fixture could not drive it.

By repository: community (archive-listed): 19 samples from 8 repositories, 3 ran; 1 identical to code first harvested elsewhere. The run record, sample by sample. “Ran” means executed on a synthesized input, not that the code is correct or reproduces the paper.

47 repositories listed; official and paper-mentioned ones first.

545641826/espcn mentioned on GitHubpytorch report
Araxeus/PNG-Upscale mentioned on GitHubtfMIT report
HighVoltageRocknRoll/sr mentioned on GitHubtfMIT report
KeremTurgutlu/papers mentioned on GitHub report
Lornatang/ESPCN-PyTorch mentioned on GitHubpytorch report
Nhat-Thanh/ESPCN-Pytorch mentioned on GitHubpytorchMIT report
Nhat-Thanh/ESPCN-TF mentioned on GitHubtfMIT report
One-sixth/pixelshuffle_invert_pytorch mentioned on GitHubpytorch report
TahmasbiM/Example mentioned on GitHubtf report
TomokiKomiya/SRGAN-keras mentioned on GitHubtf report
Zhangyanbo/iResNetLab mentioned on GitHubpytorch report
alexjc/neural-enhance mentioned on GitHub report
anujdutt9/ESPCN mentioned on GitHubpytorchMIT report
atriumlts/subpixel mentioned on GitHubtfMIT report
deepak112/Keras-SRGAN mentioned on GitHubtf report
fengwang/subpixel_conv2d mentioned on GitHubtf report
gs18113/ESPCN-TensorFlow2 mentioned on GitHubtf report
jaingaurav3/GAN-Hacks mentioned on GitHub report
kingcheng2000/GAN mentioned on GitHub report
leftthomas/espcn mentioned on GitHubpytorch report
linxi159/GAN-training-tricks mentioned on GitHub report
med-seg/kidney mentioned on GitHubtf report
med-seg/kidney-mc mentioned on GitHubtf report
michael13162/DoodleGAN mentioned on GitHub report
niazwazir/REAL_TIME_VIDEO_IMSR mentioned on GitHubpytorchMIT report
npielawski/pytorch_tiramisu mentioned on GitHubpytorchMIT report
poikilos/pyrotocanvas mentioned on GitHubtfGPL-3.0 report
quin-med-harvard-edu/kidney mentioned on GitHubtf report
ssulun/pytorch-pixelshuffle1d mentioned on GitHubpytorch report
tetrachrome/subpixel mentioned on GitHubtfMIT report
twhui/SRGAN-PyTorch mentioned on GitHubpytorch report
vuanhtu1993/Keras-SRGANs mentioned on GitHubtf report
pwc-1/Paper-9 mindspore report
pwc-1/Paper-9 mindspore report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

20 samples harvested; 4 ran; 0 honoured the contract we drafted; 16 have no recorded run. Read from Syntology's graph 2026-09-24; that is when this build read the record, not when the samples ran.

2ran · our draft was wrong
2ran · fixture could not drive it
16unverified

Licence: 1 of the 20 samples is pointer only, meaning Syntology does not serve that copy's text. This page shows no code text for any sample; each one links to its file in the repository.

Harvested from 8 repositories linked to this paper, official or community; each sample names its own and says which. Some samples are identical code Syntology first harvested from another repository; for those, this paper's copy is not located and its licence is not recorded. “Ran” means the sample executed on a synthesized input. It does not mean the output is correct, and nothing here reproduces the paper's results. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself; “our draft was wrong” and “fixture could not drive it” are failures of Syntology's instrument, not of the code.

Each sample ends with its code_sha256, Syntology's identity for that exact code. An agent fetches the stored sample with Syntology's MCP tool get_code(code_sha256="…") (how to connect); click an identity to copy that call.

Repository labels, per sample. official repository: The archive marks this repository official for the paper. named in the paper: The archive records that the paper mentions this repository; it is not marked official. community (archive-listed): In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper. found in paper text by Syntology: Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted. community: Not in the archive's code links for this paper; a community repository Syntology harvested. Samples from a repository marked official are listed first. Licence labels name the repository's licence as recorded at harvest. “Pointer only” means Syntology does not serve that copy's text, for one of four reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; or the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar). Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence label for the reason. File links open the file on GitHub at the default branch, which may have changed since the harvest.

choice_device Lornatang/ESPCN-PyTorch/inference.py community (archive-listed) ran · our draft was wrong Apache-2.0 (permissive) · 3ddf550638922b31 · report
pixelshuffle One-sixth/pixelshuffle_invert_pytorch/pixelshuffle_invert.py community (archive-listed) ran · fixture could not drive it MIT (permissive) · 8c2feff80635400e · report
pixelshuffle_invert One-sixth/pixelshuffle_invert_pytorch/pixelshuffle_invert.py community (archive-listed) ran · fixture could not drive it MIT (permissive) · 85cdae1b9119a1aa · report
PSNR behnoudshafizadeh/ESPCN-SuperResolution-Tensorflow/utils.py community (archive-listed) unverified Apache-2.0 (permissive) · 5a900d242c89ecf1 · report
bytes_feature HighVoltageRocknRoll/sr/datasets/prepare_dataset.py community (archive-listed) unverified MIT (permissive) · d682a9593710857e · report
calculate_valid_crop_size niazwazir/REAL_TIME_VIDEO_IMSR/data_utils.py community (archive-listed) unverified MIT (permissive) · c889b417f65ffc2a · report
center_crop npielawski/pytorch_tiramisu/tiramisu/tiramisu.py community (archive-listed) unverified MIT (permissive) · fbf4fd83b502b493 · report
crop_asym_like npielawski/pytorch_tiramisu/tiramisu/tiramisu.py community (archive-listed) unverified MIT (permissive) · 559b2398ad70b2dd · report
image_warp HighVoltageRocknRoll/sr/models/image_warp.py community (archive-listed) unverified MIT (permissive) · 307b25262433fb97 · report
is_image_file niazwazir/REAL_TIME_VIDEO_IMSR/data_utils.py community (archive-listed) unverified MIT (permissive) · 7e6cb8a85c64996e · report
is_video_file niazwazir/REAL_TIME_VIDEO_IMSR/data_utils.py community (archive-listed) unverified MIT (permissive) · b8a23b27c781a66c · report
pixel_shuffle Nhat-Thanh/ESPCN-TF/neuralnet.py community (archive-listed) unverified MIT (permissive) · c8d3363d1625adc6 · report
psnr niazwazir/REAL_TIME_VIDEO_IMSR/PSNR.py community (archive-listed) unverified MIT (permissive) · 440d24ec30b09c77 · report
read_image Nhat-Thanh/ESPCN-Pytorch/utils/common.py community (archive-listed) unverified MIT (permissive) · 4d448d32e15f04c9 · report
read_image Nhat-Thanh/ESPCN-TF/utils/common.py community (archive-listed) unverified MIT (permissive) · 05f12c47fb9b8978 · report
rgb2ycbcr Nhat-Thanh/ESPCN-Pytorch/utils/common.py community (archive-listed) unverified MIT (permissive) · d8058eded6a9e999 · report
rgb2ycbcr Nhat-Thanh/ESPCN-TF/utils/common.py community (archive-listed) unverified MIT (permissive) · 3760ee509acdf19f · report
ycbcr2rgb Nhat-Thanh/ESPCN-Pytorch/utils/common.py community (archive-listed) unverified MIT (permissive) · ef42ea280e97a9f8 · report
ycbcr2rgb Nhat-Thanh/ESPCN-TF/utils/common.py community (archive-listed) unverified MIT (permissive) · 3b0edbcfb724a432 · report
extend identical code first harvested elsewhere ran · our draft was wrong licence of this copy not recorded · a8dabd6fc5e7e33d · report

Tasks

Image Super-ResolutionSuper-ResolutionVideo Super-Resolution

1 archive task tag without a task page not shown.

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Image Super-Resolution BSD100 - 4x upscaling ESPCN MOS 2.01 #53 of 71 Archive leaderboard report
Image Super-Resolution BSD100 - 4x upscaling ESPCN PSNR 27.02 #53 of 71 Archive leaderboard report
Image Super-Resolution BSD100 - 4x upscaling ESPCN SSIM 0.7442 #53 of 71 Archive leaderboard report
Image Super-Resolution Set14 - 4x upscaling ESPCN MOS 2.52 #88 of 104 Archive leaderboard report
Image Super-Resolution Set14 - 4x upscaling ESPCN PSNR 27.66 #88 of 104 Archive leaderboard report
Image Super-Resolution Set14 - 4x upscaling ESPCN SSIM 0.8004 #88 of 104 Archive leaderboard report
Video Super-Resolution MSU Video Super Resolution Benchmark: Detail Restoration ESPCN 1 - LPIPS 0.765 #31 of 32 Archive leaderboard report
Video Super-Resolution MSU Video Super Resolution Benchmark: Detail Restoration ESPCN ERQAv1.0 0.521 #31 of 32 Archive leaderboard report
Video Super-Resolution MSU Video Super Resolution Benchmark: Detail Restoration ESPCN FPS 3.333 #31 of 32 Archive leaderboard report
Video Super-Resolution MSU Video Super Resolution Benchmark: Detail Restoration ESPCN PSNR 26.714 #31 of 32 Archive leaderboard report
Video Super-Resolution MSU Video Super Resolution Benchmark: Detail Restoration ESPCN QRCRv1.0 0 #31 of 32 Archive leaderboard report
Video Super-Resolution MSU Video Super Resolution Benchmark: Detail Restoration ESPCN SSIM 0.811 #31 of 32 Archive leaderboard report
Video Super-Resolution MSU Video Super Resolution Benchmark: Detail Restoration ESPCN Subjective score 2.099 #31 of 32 Archive leaderboard report
Video Super-Resolution MSU Video Upscalers: Quality Enhancement ESPCN PSNR 26.25 #36 of 48 Archive leaderboard report
Video Super-Resolution MSU Video Upscalers: Quality Enhancement ESPCN SSIM 0.926 #36 of 48 Archive leaderboard report
Video Super-Resolution MSU Video Upscalers: Quality Enhancement ESPCN VMAF 47.19 #36 of 48 Archive leaderboard report
Video Super-Resolution Ultra Video Group HD - 4x upscaling ESPCN Average PSNR 37.91 #3 of 5 Archive leaderboard report
Video Super-Resolution Ultra Video Group HD - 4x upscaling bicubic Average PSNR 36.20 #5 of 5 Archive leaderboard report
Video Super-Resolution Vid4 - 4x upscaling ESPCN MOVIE 6.54 #20 of 27 Archive leaderboard report
Video Super-Resolution Vid4 - 4x upscaling ESPCN PSNR 25.06 #20 of 27 Archive leaderboard report
Video Super-Resolution Vid4 - 4x upscaling ESPCN SSIM 0.7394 #20 of 27 Archive leaderboard report
Video Super-Resolution Xiph HD - 4x upscaling ESPCN Average PSNR 31.67 #1 of 3 Archive leaderboard report
Video Super-Resolution Xiph HD - 4x upscaling bicubic Average PSNR 30.30 #3 of 3 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Introduced by this paper: PixelShuffle

ConvolutionPixelShuffle

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections