{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/zits-image-inpainting-by-improving-the","title":"ZITS++: Image Inpainting by Improving the Incremental Transformer on Structural Priors","arxiv_id":"2210.05950","date":"2022-10-12","proceeding":null,"authors":["Chenjie Cao","Qiaole Dong","Yanwei Fu"],"abstract":"Image inpainting involves filling missing areas of a corrupted image. Despite impressive results have been achieved recently, restoring images with both vivid textures and reasonable structures remains a significant challenge. Previous methods have primarily addressed regular textures while disregarding holistic structures due to the limited receptive fields of Convolutional Neural Networks (CNNs). To this end, we study learning a Zero-initialized residual addition based Incremental Transformer on Structural priors (ZITS++), an improved model upon our conference work, ZITS. Specifically, given one corrupt image, we present the Transformer Structure Restorer (TSR) module to restore holistic structural priors at low image resolution, which are further upsampled by Simple Structure Upsampler (SSU) module to higher image resolution. To recover image texture details, we use the Fourier CNN Texture Restoration (FTR) module, which is strengthened by Fourier and large-kernel attention convolutions. Furthermore, to enhance the FTR, the upsampled structural priors from TSR are further processed by Structure Feature Encoder (SFE) and optimized with the Zero-initialized Residual Addition (ZeroRA) incrementally. Besides, a new masking positional encoding is proposed to encode the large irregular masks. Compared with ZITS, ZITS++ improves the FTR's stability and inpainting ability with several techniques. More importantly, we comprehensively explore the effects of various image priors for inpainting and investigate how to utilize them to address high-resolution image inpainting with extensive experiments. This investigation is orthogonal to most inpainting approaches and can thus significantly benefit the community. Codes and models will be released in https://github.com/ewrfcas/ZITS-PlusPlus.","url_abs":"https://arxiv.org/abs/2210.05950v3","url_pdf":"https://arxiv.org/pdf/2210.05950v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"zits-image-inpainting-by-improving-the","repo_url":"https://github.com/ewrfcas/zits-plusplus","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"zits-image-inpainting-by-improving-the","repo_url":"https://github.com/dqiaole/zits_inpainting","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"image-inpainting","task_name":"Image Inpainting"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"pixel-prediction","method_name":"Inpainting"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2210.05950","atlas_url":"https://app.syntology.ai/?focus=2210.05950","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.05950"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ewrfcas/zits-plusplus","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dqiaole/zits_inpainting","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":6,"unverified":3},"by_repo_kind":{"official":{"samples":9,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1c2fb75a32c87a57","entry":"calculate_activation_statistics","repo":"ewrfcas/zits-plusplus","repo_kind":"official","path":"inpainting_metric.py","file_url":"https://github.com/ewrfcas/zits-plusplus/blob/HEAD/inpainting_metric.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1c2fb75a32c87a57"}},{"code_sha256_prefix":"d059051e90165ebe","entry":"calculate_frechet_distance","repo":"ewrfcas/zits-plusplus","repo_kind":"official","path":"inpainting_metric.py","file_url":"https://github.com/ewrfcas/zits-plusplus/blob/HEAD/inpainting_metric.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d059051e90165ebe"}},{"code_sha256_prefix":"1ec3907b9c7cefae","entry":"feature_matching_loss","repo":"ewrfcas/zits-plusplus","repo_kind":"official","path":"networks/losses.py","file_url":"https://github.com/ewrfcas/zits-plusplus/blob/HEAD/networks/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1ec3907b9c7cefae"}},{"code_sha256_prefix":"7ccf005b1c05d4da","entry":"generator_loss","repo":"ewrfcas/zits-plusplus","repo_kind":"official","path":"networks/losses.py","file_url":"https://github.com/ewrfcas/zits-plusplus/blob/HEAD/networks/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7ccf005b1c05d4da"}},{"code_sha256_prefix":"18bfc140d0a26202","entry":"get_activations","repo":"ewrfcas/zits-plusplus","repo_kind":"official","path":"inpainting_metric.py","file_url":"https://github.com/ewrfcas/zits-plusplus/blob/HEAD/inpainting_metric.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"18bfc140d0a26202"}},{"code_sha256_prefix":"7d0746f0685aabac","entry":"interpolate_mask","repo":"ewrfcas/zits-plusplus","repo_kind":"official","path":"networks/losses.py","file_url":"https://github.com/ewrfcas/zits-plusplus/blob/HEAD/networks/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7d0746f0685aabac"}},{"code_sha256_prefix":"7831601a1f93f0b6","entry":"feature2token","repo":"ewrfcas/zits-plusplus","repo_kind":"official","path":"networks/mat.py","file_url":"https://github.com/ewrfcas/zits-plusplus/blob/HEAD/networks/mat.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7831601a1f93f0b6"}},{"code_sha256_prefix":"b189ed0149b76880","entry":"spectral_norm","repo":"ewrfcas/zits-plusplus","repo_kind":"official","path":"networks/layers.py","file_url":"https://github.com/ewrfcas/zits-plusplus/blob/HEAD/networks/layers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b189ed0149b76880"}},{"code_sha256_prefix":"e9a442dad357d4a7","entry":"token2feature","repo":"ewrfcas/zits-plusplus","repo_kind":"official","path":"networks/mat.py","file_url":"https://github.com/ewrfcas/zits-plusplus/blob/HEAD/networks/mat.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e9a442dad357d4a7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}