{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/incremental-transformer-structure-enhanced","title":"Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding","arxiv_id":"2203.00867","date":"2022-03-02","proceeding":"CVPR 2022 1","authors":["Qiaole Dong","Chenjie Cao","Yanwei Fu"],"abstract":"Image inpainting has made significant advances in recent years. However, it is still challenging to recover corrupted images with both vivid textures and reasonable structures. Some specific methods only tackle regular textures while losing holistic structures due to the limited receptive fields of convolutional neural networks (CNNs). On the other hand, attention-based models can learn better long-range dependency for the structure recovery, but they are limited by the heavy computation for inference with large image sizes. To address these issues, we propose to leverage an additional structure restorer to facilitate the image inpainting incrementally. The proposed model restores holistic image structures with a powerful attention-based transformer model in a fixed low-resolution sketch space. Such a grayscale space is easy to be upsampled to larger scales to convey correct structural information. Our structure restorer can be integrated with other pretrained inpainting models efficiently with the zero-initialized residual addition. Furthermore, a masking positional encoding strategy is utilized to improve the performance with large irregular masks. Extensive experiments on various datasets validate the efficacy of our model compared with other competitors. Our codes are released in https://github.com/DQiaole/ZITS_inpainting.","url_abs":"https://arxiv.org/abs/2203.00867v2","url_pdf":"https://arxiv.org/pdf/2203.00867v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"incremental-transformer-structure-enhanced","repo_url":"https://github.com/dqiaole/zits_inpainting","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"incremental-transformer-structure-enhanced","repo_url":"https://github.com/dqiaole/zits","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"image-inpainting","task_name":"Image Inpainting"}],"methods":[{"method_slug":"pixel-prediction","method_name":"Inpainting"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.00867","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.00867"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dqiaole/zits_inpainting","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dqiaole/zits","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/DQiaole/ZITS_inpainting","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":5,"ran_honours":1,"unverified":10},"by_repo_kind":{"official":{"samples":9,"ran":6,"repositories":1},"listed":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"374b0040c780395d","entry":"CausalSelfAttention","repo":"DQiaole/ZITS_inpainting","repo_kind":"official","path":"src/models/TSR_model.py","file_url":"https://github.com/DQiaole/ZITS_inpainting/blob/HEAD/src/models/TSR_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"374b0040c780395d"}},{"code_sha256_prefix":"c088eb9584bb3903","entry":"GELU","repo":"DQiaole/ZITS_inpainting","repo_kind":"official","path":"src/models/TSR_model.py","file_url":"https://github.com/DQiaole/ZITS_inpainting/blob/HEAD/src/models/TSR_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c088eb9584bb3903"}},{"code_sha256_prefix":"0dcf0903f8538d11","entry":"GELU2","repo":"DQiaole/ZITS_inpainting","repo_kind":"official","path":"src/models/TSR_model.py","file_url":"https://github.com/DQiaole/ZITS_inpainting/blob/HEAD/src/models/TSR_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0dcf0903f8538d11"}},{"code_sha256_prefix":"96c2a9a4d1843cec","entry":"SelfAttention","repo":"DQiaole/ZITS_inpainting","repo_kind":"official","path":"src/models/TSR_model.py","file_url":"https://github.com/DQiaole/ZITS_inpainting/blob/HEAD/src/models/TSR_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"96c2a9a4d1843cec"}},{"code_sha256_prefix":"50e1ffed03f484ec","entry":"gelu","repo":"DQiaole/ZITS_inpainting","repo_kind":"official","path":"src/models/TSR_model.py","file_url":"https://github.com/DQiaole/ZITS_inpainting/blob/HEAD/src/models/TSR_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"50e1ffed03f484ec"}},{"code_sha256_prefix":"5db9dac361c10f1f","entry":"my_Block_2","repo":"DQiaole/ZITS_inpainting","repo_kind":"official","path":"src/models/TSR_model.py","file_url":"https://github.com/DQiaole/ZITS_inpainting/blob/HEAD/src/models/TSR_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5db9dac361c10f1f"}},{"code_sha256_prefix":"79e5b2a4b67a561a","entry":"AxialAttention","repo":"DQiaole/ZITS_inpainting","repo_kind":"official","path":"src/models/TSR_model.py","file_url":"https://github.com/DQiaole/ZITS_inpainting/blob/HEAD/src/models/TSR_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"79e5b2a4b67a561a"}},{"code_sha256_prefix":"fba8e898c2b6bfd4","entry":"BlockAxial","repo":"DQiaole/ZITS_inpainting","repo_kind":"official","path":"src/models/TSR_model.py","file_url":"https://github.com/DQiaole/ZITS_inpainting/blob/HEAD/src/models/TSR_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fba8e898c2b6bfd4"}},{"code_sha256_prefix":"cdf9104f2e54ae72","entry":"EdgeLineGPT256RelBCE","repo":"DQiaole/ZITS_inpainting","repo_kind":"official","path":"src/models/TSR_model.py","file_url":"https://github.com/DQiaole/ZITS_inpainting/blob/HEAD/src/models/TSR_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cdf9104f2e54ae72"}},{"code_sha256_prefix":"42c0844d8edaf70f","entry":"SampleEdgeLineLogits","repo":"dqiaole/zits","repo_kind":"listed","path":"src/utils.py","file_url":"https://github.com/dqiaole/zits/blob/HEAD/src/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"42c0844d8edaf70f"}},{"code_sha256_prefix":"1c2fb75a32c87a57","entry":"calculate_activation_statistics","repo":"dqiaole/zits","repo_kind":"listed","path":"src/inpainting_metrics.py","file_url":"https://github.com/dqiaole/zits/blob/HEAD/src/inpainting_metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1c2fb75a32c87a57"}},{"code_sha256_prefix":"d059051e90165ebe","entry":"calculate_frechet_distance","repo":"dqiaole/zits","repo_kind":"listed","path":"src/inpainting_metrics.py","file_url":"https://github.com/dqiaole/zits/blob/HEAD/src/inpainting_metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d059051e90165ebe"}},{"code_sha256_prefix":"817b9a0ad441fb50","entry":"create_mask","repo":"dqiaole/zits","repo_kind":"listed","path":"src/utils.py","file_url":"https://github.com/dqiaole/zits/blob/HEAD/src/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"817b9a0ad441fb50"}},{"code_sha256_prefix":"18bfc140d0a26202","entry":"get_activations","repo":"dqiaole/zits","repo_kind":"listed","path":"src/inpainting_metrics.py","file_url":"https://github.com/dqiaole/zits/blob/HEAD/src/inpainting_metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"18bfc140d0a26202"}},{"code_sha256_prefix":"9ff337bc4bf82da5","entry":"stitch_images","repo":"dqiaole/zits","repo_kind":"listed","path":"src/utils.py","file_url":"https://github.com/dqiaole/zits/blob/HEAD/src/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9ff337bc4bf82da5"}},{"code_sha256_prefix":"bad108729054f08a","entry":"to_int","repo":"dqiaole/zits","repo_kind":"listed","path":"datasets/dataset_FTR.py","file_url":"https://github.com/dqiaole/zits/blob/HEAD/datasets/dataset_FTR.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bad108729054f08a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}