{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/realviformer-investigating-attention-for-real","title":"RealViformer: Investigating Attention for Real-World Video Super-Resolution","arxiv_id":"2407.13987","date":"2024-07-19","proceeding":null,"authors":["Yuehan Zhang","Angela Yao"],"abstract":"In real-world video super-resolution (VSR), videos suffer from in-the-wild degradations and artifacts. VSR methods, especially recurrent ones, tend to propagate artifacts over time in the real-world setting and are more vulnerable than image super-resolution. This paper investigates the influence of artifacts on commonly used covariance-based attention mechanisms in VSR. Comparing the widely-used spatial attention, which computes covariance over space, versus the channel attention, we observe that the latter is less sensitive to artifacts. However, channel attention leads to feature redundancy, as evidenced by the higher covariance among output channels. As such, we explore simple techniques such as the squeeze-excite mechanism and covariance-based rescaling to counter the effects of high channel covariance. Based on our findings, we propose RealViformer. This channel-attention-based real-world VSR framework surpasses state-of-the-art on two real-world VSR datasets with fewer parameters and faster runtimes. The source code is available at https://github.com/Yuehan717/RealViformer.","url_abs":"https://arxiv.org/abs/2407.13987v1","url_pdf":"https://arxiv.org/pdf/2407.13987v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"realviformer-investigating-attention-for-real","repo_url":"https://github.com/yuehan717/realviformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-super-resolution","task_name":"Image Super-Resolution"},{"task_slug":"super-resolution","task_name":"Super-Resolution"},{"task_slug":"video-super-resolution","task_name":"Video Super-Resolution"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2407.13987","atlas_url":"https://app.syntology.ai/?focus=2407.13987","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.13987"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yuehan717/realviformer","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/Yuehan717/RealViformer","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":14,"ran_draft_wrong":2,"ran_fixture":2,"unverified":5},"by_repo_kind":{"official":{"samples":23,"ran":18,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c673f82c27d694b0","entry":"Attention","repo":"Yuehan717/RealViformer","repo_kind":"official","path":"archs/realviformer_arch.py","file_url":"https://github.com/Yuehan717/RealViformer/blob/HEAD/archs/realviformer_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c673f82c27d694b0"}},{"code_sha256_prefix":"34544c8f80a989ea","entry":"BasicModule","repo":"Yuehan717/RealViformer","repo_kind":"official","path":"archs/realviformer_arch.py","file_url":"https://github.com/Yuehan717/RealViformer/blob/HEAD/archs/realviformer_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"34544c8f80a989ea"}},{"code_sha256_prefix":"7e2a41bb01b94fc9","entry":"BiasFree_LayerNorm","repo":"Yuehan717/RealViformer","repo_kind":"official","path":"archs/realviformer_arch.py","file_url":"https://github.com/Yuehan717/RealViformer/blob/HEAD/archs/realviformer_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7e2a41bb01b94fc9"}},{"code_sha256_prefix":"f3e998138ec90085","entry":"CrossChannelAttention","repo":"Yuehan717/RealViformer","repo_kind":"official","path":"archs/realviformer_arch.py","file_url":"https://github.com/Yuehan717/RealViformer/blob/HEAD/archs/realviformer_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f3e998138ec90085"}},{"code_sha256_prefix":"1e98c6126e991a81","entry":"FeedForward","repo":"Yuehan717/RealViformer","repo_kind":"official","path":"archs/realviformer_arch.py","file_url":"https://github.com/Yuehan717/RealViformer/blob/HEAD/archs/realviformer_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1e98c6126e991a81"}},{"code_sha256_prefix":"f65e81cd69bbf677","entry":"LayerNorm","repo":"Yuehan717/RealViformer","repo_kind":"official","path":"archs/realviformer_arch.py","file_url":"https://github.com/Yuehan717/RealViformer/blob/HEAD/archs/realviformer_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f65e81cd69bbf677"}},{"code_sha256_prefix":"1098bd700b316e9f","entry":"SpyNet","repo":"Yuehan717/RealViformer","repo_kind":"official","path":"archs/realviformer_arch.py","file_url":"https://github.com/Yuehan717/RealViformer/blob/HEAD/archs/realviformer_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1098bd700b316e9f"}},{"code_sha256_prefix":"dc176fc0afbcaf34","entry":"Upsample","repo":"Yuehan717/RealViformer","repo_kind":"official","path":"archs/realviformer_arch.py","file_url":"https://github.com/Yuehan717/RealViformer/blob/HEAD/archs/realviformer_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dc176fc0afbcaf34"}},{"code_sha256_prefix":"a19b3d60ae1aaffa","entry":"WithBias_LayerNorm","repo":"Yuehan717/RealViformer","repo_kind":"official","path":"archs/realviformer_arch.py","file_url":"https://github.com/Yuehan717/RealViformer/blob/HEAD/archs/realviformer_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a19b3d60ae1aaffa"}},{"code_sha256_prefix":"ef9faf68f492a375","entry":"flow_warp","repo":"Yuehan717/RealViformer","repo_kind":"official","path":"archs/realviformer_arch.py","file_url":"https://github.com/Yuehan717/RealViformer/blob/HEAD/archs/realviformer_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ef9faf68f492a375"}},{"code_sha256_prefix":"8b5cf3d00508f17b","entry":"img2tensor","repo":"yuehan717/realviformer","repo_kind":"official","path":"img_util.py","file_url":"https://github.com/yuehan717/realviformer/blob/HEAD/img_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8b5cf3d00508f17b"}},{"code_sha256_prefix":"96ad5dc9ca239aec","entry":"make_layer","repo":"yuehan717/realviformer","repo_kind":"official","path":"archs/arch_util.py","file_url":"https://github.com/yuehan717/realviformer/blob/HEAD/archs/arch_util.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"96ad5dc9ca239aec"}},{"code_sha256_prefix":"9f330f530563e78f","entry":"resize_flow","repo":"yuehan717/realviformer","repo_kind":"official","path":"archs/arch_util.py","file_url":"https://github.com/yuehan717/realviformer/blob/HEAD/archs/arch_util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f330f530563e78f"}},{"code_sha256_prefix":"f4e0ddb8e63a63ee","entry":"scandir","repo":"yuehan717/realviformer","repo_kind":"official","path":"data_util.py","file_url":"https://github.com/yuehan717/realviformer/blob/HEAD/data_util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f4e0ddb8e63a63ee"}},{"code_sha256_prefix":"6d6227e782335df3","entry":"tensor2img","repo":"yuehan717/realviformer","repo_kind":"official","path":"img_util.py","file_url":"https://github.com/yuehan717/realviformer/blob/HEAD/img_util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6d6227e782335df3"}},{"code_sha256_prefix":"442bf3d188b77bf1","entry":"tensor2img_fast","repo":"yuehan717/realviformer","repo_kind":"official","path":"img_util.py","file_url":"https://github.com/yuehan717/realviformer/blob/HEAD/img_util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"442bf3d188b77bf1"}},{"code_sha256_prefix":"82a15cc1e46f7e4d","entry":"to_3d","repo":"yuehan717/realviformer","repo_kind":"official","path":"archs/realviformer_arch.py","file_url":"https://github.com/yuehan717/realviformer/blob/HEAD/archs/realviformer_arch.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"82a15cc1e46f7e4d"}},{"code_sha256_prefix":"b20f2a5df739a59e","entry":"to_4d","repo":"yuehan717/realviformer","repo_kind":"official","path":"archs/realviformer_arch.py","file_url":"https://github.com/yuehan717/realviformer/blob/HEAD/archs/realviformer_arch.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b20f2a5df739a59e"}},{"code_sha256_prefix":"181873d6f2194b59","entry":"ChannelAttentionBlock","repo":"Yuehan717/RealViformer","repo_kind":"official","path":"archs/realviformer_arch.py","file_url":"https://github.com/Yuehan717/RealViformer/blob/HEAD/archs/realviformer_arch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"181873d6f2194b59"}},{"code_sha256_prefix":"4bde66e2e3722ca8","entry":"Downsample","repo":"Yuehan717/RealViformer","repo_kind":"official","path":"archs/realviformer_arch.py","file_url":"https://github.com/Yuehan717/RealViformer/blob/HEAD/archs/realviformer_arch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4bde66e2e3722ca8"}},{"code_sha256_prefix":"8378fd449c8fc540","entry":"RealViformer","repo":"Yuehan717/RealViformer","repo_kind":"official","path":"archs/realviformer_arch.py","file_url":"https://github.com/Yuehan717/RealViformer/blob/HEAD/archs/realviformer_arch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8378fd449c8fc540"}},{"code_sha256_prefix":"e91c5e9a654fd7dc","entry":"TransformerBlock","repo":"Yuehan717/RealViformer","repo_kind":"official","path":"archs/realviformer_arch.py","file_url":"https://github.com/Yuehan717/RealViformer/blob/HEAD/archs/realviformer_arch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e91c5e9a654fd7dc"}},{"code_sha256_prefix":"223d0788b0725dfb","entry":"mod_crop","repo":"yuehan717/realviformer","repo_kind":"official","path":"data_util.py","file_url":"https://github.com/yuehan717/realviformer/blob/HEAD/data_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"223d0788b0725dfb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}