{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/recurrent-video-restoration-transformer-with","title":"Recurrent Video Restoration Transformer with Guided Deformable Attention","arxiv_id":"2206.02146","date":"2022-06-05","proceeding":null,"authors":["Jingyun Liang","Yuchen Fan","Xiaoyu Xiang","Rakesh Ranjan","Eddy Ilg","Simon Green","JieZhang Cao","Kai Zhang","Radu Timofte","Luc van Gool"],"abstract":"Video restoration aims at restoring multiple high-quality frames from multiple low-quality frames. Existing video restoration methods generally fall into two extreme cases, i.e., they either restore all frames in parallel or restore the video frame by frame in a recurrent way, which would result in different merits and drawbacks. Typically, the former has the advantage of temporal information fusion. However, it suffers from large model size and intensive memory consumption; the latter has a relatively small model size as it shares parameters across frames; however, it lacks long-range dependency modeling ability and parallelizability. In this paper, we attempt to integrate the advantages of the two cases by proposing a recurrent video restoration transformer, namely RVRT. RVRT processes local neighboring frames in parallel within a globally recurrent framework which can achieve a good trade-off between model size, effectiveness, and efficiency. Specifically, RVRT divides the video into multiple clips and uses the previously inferred clip feature to estimate the subsequent clip feature. Within each clip, different frame features are jointly updated with implicit feature aggregation. Across different clips, the guided deformable attention is designed for clip-to-clip alignment, which predicts multiple relevant locations from the whole inferred clip and aggregates their features by the attention mechanism. Extensive experiments on video super-resolution, deblurring, and denoising show that the proposed RVRT achieves state-of-the-art performance on benchmark datasets with balanced model size, testing memory and runtime.","url_abs":"https://arxiv.org/abs/2206.02146v3","url_pdf":"https://arxiv.org/pdf/2206.02146v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"recurrent-video-restoration-transformer-with","repo_url":"https://github.com/jingyunliang/rvrt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"recurrent-video-restoration-transformer-with","repo_url":"https://github.com/Ascend-Research/Turtle","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"recurrent-video-restoration-transformer-with","repo_url":"https://github.com/labshuhanggu/mia-vsr","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"recurrent-video-restoration-transformer-with","repo_url":"https://github.com/xg416/DATUM","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"analog-video-restoration","task_name":"Analog Video Restoration"},{"task_slug":"deblurring","task_name":"Deblurring"},{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"snow-removal","task_name":"Snow Removal"},{"task_slug":"super-resolution","task_name":"Super-Resolution"},{"task_slug":"video-denoising","task_name":"Video Denoising"},{"task_slug":"video-restoration","task_name":"Video Restoration"},{"task_slug":"video-super-resolution","task_name":"Video Super-Resolution"},{"task_slug":"video-deraining","task_name":"Video deraining"}],"methods":[{"method_slug":"clip","method_name":"CLIP"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/analog-video-restoration-on-tape","task":"Analog Video Restoration","dataset":"TAPE","model":"RVRT","rank_in_archive_order":4,"of":7,"metrics":{"LPIPS":"0.117","PSNR":"32.47","SSIM":"0.896","VMAF":"72.41"},"uses_additional_data":false},{"leaderboard":"/sota/deblurring-on-dvd-1","task":"Deblurring","dataset":"DVD","model":"RVRT","rank_in_archive_order":2,"of":7,"metrics":{"PSNR":"34.92","SSIM":"97.38"},"uses_additional_data":false},{"leaderboard":"/sota/video-denoising-on-davis-sigma10","task":"Video Denoising","dataset":"DAVIS sigma10","model":"RVRT","rank_in_archive_order":3,"of":6,"metrics":{"PSNR":"40.57"},"uses_additional_data":false},{"leaderboard":"/sota/video-denoising-on-davis-sigma20","task":"Video Denoising","dataset":"DAVIS sigma20","model":"RVRT","rank_in_archive_order":3,"of":7,"metrics":{"PSNR":"38.05"},"uses_additional_data":false},{"leaderboard":"/sota/video-denoising-on-davis-sigma30","task":"Video Denoising","dataset":"DAVIS sigma30","model":"RVRT","rank_in_archive_order":2,"of":7,"metrics":{"PSNR":"36.57"},"uses_additional_data":false},{"leaderboard":"/sota/video-denoising-on-davis-sigma40","task":"Video Denoising","dataset":"DAVIS sigma40","model":"RVRT","rank_in_archive_order":2,"of":8,"metrics":{"PSNR":"35.47"},"uses_additional_data":false},{"leaderboard":"/sota/video-denoising-on-davis-sigma50","task":"Video Denoising","dataset":"DAVIS sigma50","model":"RVRT","rank_in_archive_order":2,"of":8,"metrics":{"PSNR":"34.57"},"uses_additional_data":false},{"leaderboard":"/sota/video-denoising-on-set8-sigma10","task":"Video Denoising","dataset":"Set8 sigma10","model":"RVRT","rank_in_archive_order":3,"of":6,"metrics":{"PSNR":"37.53"},"uses_additional_data":false},{"leaderboard":"/sota/video-denoising-on-set8-sigma20","task":"Video Denoising","dataset":"Set8 sigma20","model":"RVRT","rank_in_archive_order":3,"of":7,"metrics":{"PSNR":"34.83"},"uses_additional_data":false},{"leaderboard":"/sota/video-denoising-on-set8-sigma30","task":"Video Denoising","dataset":"Set8 sigma30","model":"RVRT","rank_in_archive_order":3,"of":7,"metrics":{"PSNR":"33.3"},"uses_additional_data":false},{"leaderboard":"/sota/video-denoising-on-set8-sigma40","task":"Video Denoising","dataset":"Set8 sigma40","model":"RVRT","rank_in_archive_order":2,"of":8,"metrics":{"PSNR":"32.21"},"uses_additional_data":false},{"leaderboard":"/sota/video-denoising-on-set8-sigma50","task":"Video Denoising","dataset":"Set8 sigma50","model":"RVRT","rank_in_archive_order":2,"of":9,"metrics":{"PSNR":"31.33"},"uses_additional_data":false},{"leaderboard":"/sota/video-super-resolution-on-udm10-4x-upscaling","task":"Video Super-Resolution","dataset":"UDM10 - 4x upscaling","model":"RVRT","rank_in_archive_order":2,"of":7,"metrics":{"PSNR":"40.9","SSIM":"0.9729"},"uses_additional_data":false},{"leaderboard":"/sota/video-super-resolution-on-vid4-4x-upscaling","task":"Video Super-Resolution","dataset":"Vid4 - 4x upscaling","model":"RVRT","rank_in_archive_order":7,"of":27,"metrics":{"PSNR":"27.99","SSIM":"0.8462"},"uses_additional_data":false},{"leaderboard":"/sota/video-super-resolution-on-vid4-4x-upscaling-1","task":"Video Super-Resolution","dataset":"Vid4 - 4x upscaling - BD degradation","model":"RVRT","rank_in_archive_order":1,"of":18,"metrics":{"PSNR":"29.54","SSIM":"0.8810"},"uses_additional_data":false},{"leaderboard":"/sota/video-super-resolution-on-vimeo90k","task":"Video Super-Resolution","dataset":"Vimeo90K","model":"RVRT","rank_in_archive_order":2,"of":3,"metrics":{"PSNR":"38.59","SSIM":"0.9576"},"uses_additional_data":false},{"leaderboard":"/sota/video-deraining-on-vrds","task":"Video deraining","dataset":"VRDS","model":"RVRT","rank_in_archive_order":8,"of":8,"metrics":{"PSNR":"28.24","SSIM":"0.8857"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2206.02146","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.02146"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/JingyunLiang/RVRT","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Ascend-Research/Turtle","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xg416/DATUM","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/labshuhanggu/mia-vsr","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jingyunliang/rvrt","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_draft_wrong":5,"ran_fixture":2,"unverified":7},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1},"listed":{"samples":9,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"7839fb1aca0933e2","entry":"batch_index_fill","repo":"labshuhanggu/mia-vsr","repo_kind":"listed","path":"archs/functions.py","file_url":"https://github.com/labshuhanggu/mia-vsr/blob/HEAD/archs/functions.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7839fb1aca0933e2"}},{"code_sha256_prefix":"52d96aa31ed74a56","entry":"drop_path","repo":"labshuhanggu/mia-vsr","repo_kind":"listed","path":"archs/mia_sliding_arch.py","file_url":"https://github.com/labshuhanggu/mia-vsr/blob/HEAD/archs/mia_sliding_arch.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"52d96aa31ed74a56"}},{"code_sha256_prefix":"0c02fb4e80bda186","entry":"flow_warp","repo":"JingyunLiang/RVRT","repo_kind":"official","path":"models/network_rvrt.py","file_url":"https://github.com/JingyunLiang/RVRT/blob/HEAD/models/network_rvrt.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"0c02fb4e80bda186"}},{"code_sha256_prefix":"9e66eec08eeb6369","entry":"flow_warp_avg_patch","repo":"labshuhanggu/mia-vsr","repo_kind":"listed","path":"archs/mia_vsr_arch.py","file_url":"https://github.com/labshuhanggu/mia-vsr/blob/HEAD/archs/mia_vsr_arch.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9e66eec08eeb6369"}},{"code_sha256_prefix":"b80adc256b420252","entry":"make_layer","repo":"JingyunLiang/RVRT","repo_kind":"official","path":"models/network_rvrt.py","file_url":"https://github.com/JingyunLiang/RVRT/blob/HEAD/models/network_rvrt.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"b80adc256b420252"}},{"code_sha256_prefix":"a1cad82b9beff0c7","entry":"test_video","repo":"JingyunLiang/RVRT","repo_kind":"official","path":"main_test_rvrt.py","file_url":"https://github.com/JingyunLiang/RVRT/blob/HEAD/main_test_rvrt.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"a1cad82b9beff0c7"}},{"code_sha256_prefix":"54da2d24c1765483","entry":"window_partition","repo":"JingyunLiang/RVRT","repo_kind":"official","path":"models/network_rvrt.py","file_url":"https://github.com/JingyunLiang/RVRT/blob/HEAD/models/network_rvrt.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"54da2d24c1765483"}},{"code_sha256_prefix":"6bc3966389f69fe9","entry":"batch_index_select","repo":"labshuhanggu/mia-vsr","repo_kind":"listed","path":"archs/functions.py","file_url":"https://github.com/labshuhanggu/mia-vsr/blob/HEAD/archs/functions.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6bc3966389f69fe9"}},{"code_sha256_prefix":"a648a03a952822c0","entry":"reduce_loss","repo":"labshuhanggu/mia-vsr","repo_kind":"listed","path":"losses/loss_util.py","file_url":"https://github.com/labshuhanggu/mia-vsr/blob/HEAD/losses/loss_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a648a03a952822c0"}},{"code_sha256_prefix":"705758eeba9070e3","entry":"test_clip","repo":"JingyunLiang/RVRT","repo_kind":"official","path":"main_test_rvrt.py","file_url":"https://github.com/JingyunLiang/RVRT/blob/HEAD/main_test_rvrt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"705758eeba9070e3"}},{"code_sha256_prefix":"1ba39317ea81871a","entry":"weight_reduce_loss","repo":"labshuhanggu/mia-vsr","repo_kind":"listed","path":"losses/loss_util.py","file_url":"https://github.com/labshuhanggu/mia-vsr/blob/HEAD/losses/loss_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1ba39317ea81871a"}},{"code_sha256_prefix":"cf63f8afc13f62a7","entry":"weighted_loss","repo":"labshuhanggu/mia-vsr","repo_kind":"listed","path":"losses/loss_util.py","file_url":"https://github.com/labshuhanggu/mia-vsr/blob/HEAD/losses/loss_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cf63f8afc13f62a7"}},{"code_sha256_prefix":"e0f3c2fd11d92ad7","entry":"window_partition","repo":"labshuhanggu/mia-vsr","repo_kind":"listed","path":"archs/mia_sliding_arch.py","file_url":"https://github.com/labshuhanggu/mia-vsr/blob/HEAD/archs/mia_sliding_arch.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e0f3c2fd11d92ad7"}},{"code_sha256_prefix":"c0dc7488e394219c","entry":"window_single_partition","repo":"labshuhanggu/mia-vsr","repo_kind":"listed","path":"archs/mia_sliding_arch.py","file_url":"https://github.com/labshuhanggu/mia-vsr/blob/HEAD/archs/mia_sliding_arch.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c0dc7488e394219c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}