{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cross-aggregation-transformer-for-image","title":"Cross Aggregation Transformer for Image Restoration","arxiv_id":"2211.13654","date":"2022-11-24","proceeding":null,"authors":["Zheng Chen","Yulun Zhang","Jinjin Gu","Yongbing Zhang","Linghe Kong","Xin Yuan"],"abstract":"Recently, Transformer architecture has been introduced into image restoration to replace convolution neural network (CNN) with surprising results. Considering the high computational complexity of Transformer with global attention, some methods use the local square window to limit the scope of self-attention. However, these methods lack direct interaction among different windows, which limits the establishment of long-range dependencies. To address the above issue, we propose a new image restoration model, Cross Aggregation Transformer (CAT). The core of our CAT is the Rectangle-Window Self-Attention (Rwin-SA), which utilizes horizontal and vertical rectangle window attention in different heads parallelly to expand the attention area and aggregate the features cross different windows. We also introduce the Axial-Shift operation for different window interactions. Furthermore, we propose the Locality Complementary Module to complement the self-attention mechanism, which incorporates the inductive bias of CNN (e.g., translation invariance and locality) into Transformer, enabling global-local coupling. Extensive experiments demonstrate that our CAT outperforms recent state-of-the-art methods on several image restoration applications. The code and models are available at https://github.com/zhengchen1999/CAT.","url_abs":"https://arxiv.org/abs/2211.13654v2","url_pdf":"https://arxiv.org/pdf/2211.13654v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cross-aggregation-transformer-for-image","repo_url":"https://github.com/zhengchen1999/cat","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"cross-aggregation-transformer-for-image","repo_url":"https://github.com/Jiamian-Wang/IST_for_SR_pruning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"cross-aggregation-transformer-for-image","repo_url":"https://github.com/jiamian-wang/iterative-soft-shrinkage-sr","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"image-restoration","task_name":"Image Restoration"},{"task_slug":"inductive-bias","task_name":"Inductive Bias"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2211.13654","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2211.13654"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/zhengchen1999/CAT","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jiamian-wang/iterative-soft-shrinkage-sr","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Jiamian-Wang/IST_for_SR_pruning","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhengchen1999/cat","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":5,"ran_fixture":1,"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":11,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9fcd98d2097453fd","entry":"Attention_axial","repo":"zhengchen1999/cat","repo_kind":"official","path":"basicsr/archs/cat_arch.py","file_url":"https://github.com/zhengchen1999/cat/blob/HEAD/basicsr/archs/cat_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9fcd98d2097453fd"}},{"code_sha256_prefix":"69065b697deb15e0","entry":"Attention_regular","repo":"zhengchen1999/cat","repo_kind":"official","path":"basicsr/archs/cat_arch.py","file_url":"https://github.com/zhengchen1999/cat/blob/HEAD/basicsr/archs/cat_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"69065b697deb15e0"}},{"code_sha256_prefix":"705379c0ebcadf0a","entry":"DynamicPosBias","repo":"zhengchen1999/cat","repo_kind":"official","path":"basicsr/archs/cat_arch.py","file_url":"https://github.com/zhengchen1999/cat/blob/HEAD/basicsr/archs/cat_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"705379c0ebcadf0a"}},{"code_sha256_prefix":"2cd4dd2a6360b684","entry":"Registry","repo":"zhengchen1999/cat","repo_kind":"official","path":"basicsr/archs/cat_arch.py","file_url":"https://github.com/zhengchen1999/cat/blob/HEAD/basicsr/archs/cat_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2cd4dd2a6360b684"}},{"code_sha256_prefix":"eaa574d0bc3bf779","entry":"Upsample","repo":"zhengchen1999/cat","repo_kind":"official","path":"basicsr/archs/cat_arch.py","file_url":"https://github.com/zhengchen1999/cat/blob/HEAD/basicsr/archs/cat_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"eaa574d0bc3bf779"}},{"code_sha256_prefix":"15c613ecceb6aa71","entry":"img2windows","repo":"zhengchen1999/cat","repo_kind":"official","path":"basicsr/archs/cat_arch.py","file_url":"https://github.com/zhengchen1999/cat/blob/HEAD/basicsr/archs/cat_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"15c613ecceb6aa71"}},{"code_sha256_prefix":"8cb23a62748d3b1c","entry":"windows2img","repo":"zhengchen1999/cat","repo_kind":"official","path":"basicsr/archs/cat_arch.py","file_url":"https://github.com/zhengchen1999/cat/blob/HEAD/basicsr/archs/cat_arch.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8cb23a62748d3b1c"}},{"code_sha256_prefix":"b1d288dec78999a4","entry":"CAT","repo":"zhengchen1999/cat","repo_kind":"official","path":"basicsr/archs/cat_arch.py","file_url":"https://github.com/zhengchen1999/cat/blob/HEAD/basicsr/archs/cat_arch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b1d288dec78999a4"}},{"code_sha256_prefix":"022d78e3cb045a68","entry":"CATB_axial","repo":"zhengchen1999/cat","repo_kind":"official","path":"basicsr/archs/cat_arch.py","file_url":"https://github.com/zhengchen1999/cat/blob/HEAD/basicsr/archs/cat_arch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"022d78e3cb045a68"}},{"code_sha256_prefix":"4f917d55d2a0051e","entry":"CATB_regular","repo":"zhengchen1999/cat","repo_kind":"official","path":"basicsr/archs/cat_arch.py","file_url":"https://github.com/zhengchen1999/cat/blob/HEAD/basicsr/archs/cat_arch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4f917d55d2a0051e"}},{"code_sha256_prefix":"6064f631b5d071d2","entry":"ResidualGroup","repo":"zhengchen1999/cat","repo_kind":"official","path":"basicsr/archs/cat_arch.py","file_url":"https://github.com/zhengchen1999/cat/blob/HEAD/basicsr/archs/cat_arch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6064f631b5d071d2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}