{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-prior-feature-and-attention-enhanced","title":"Learning Prior Feature and Attention Enhanced Image Inpainting","arxiv_id":"2208.01837","date":"2022-08-03","proceeding":null,"authors":["Chenjie Cao","Qiaole Dong","Yanwei Fu"],"abstract":"Many recent inpainting works have achieved impressive results by leveraging Deep Neural Networks (DNNs) to model various prior information for image restoration. Unfortunately, the performance of these methods is largely limited by the representation ability of vanilla Convolutional Neural Networks (CNNs) backbones.On the other hand, Vision Transformers (ViT) with self-supervised pre-training have shown great potential for many visual recognition and object detection tasks. A natural question is whether the inpainting task can be greatly benefited from the ViT backbone? However, it is nontrivial to directly replace the new backbones in inpainting networks, as the inpainting is an inverse problem fundamentally different from the recognition tasks. To this end, this paper incorporates the pre-training based Masked AutoEncoder (MAE) into the inpainting model, which enjoys richer informative priors to enhance the inpainting process. Moreover, we propose to use attention priors from MAE to make the inpainting model learn more long-distance dependencies between masked and unmasked regions. Sufficient ablations have been discussed about the inpainting and the self-supervised pre-training models in this paper. Besides, experiments on both Places2 and FFHQ demonstrate the effectiveness of our proposed model. Codes and pre-trained models are released in https://github.com/ewrfcas/MAE-FAR.","url_abs":"https://arxiv.org/abs/2208.01837v2","url_pdf":"https://arxiv.org/pdf/2208.01837v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-prior-feature-and-attention-enhanced","repo_url":"https://github.com/ewrfcas/MAE-FAR","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-inpainting","task_name":"Image Inpainting"},{"task_slug":"image-restoration","task_name":"Image Restoration"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"pixel-prediction","method_name":"Inpainting"},{"method_slug":"mae","method_name":"MAE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2208.01837","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2208.01837"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/facebookresearch/mae","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/ewrfcas/MAE-FAR","reach":null}],"summary":{"ran_fixture":1,"unverified":9},"by_repo_kind":{"official":{"samples":10,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"d1f2e1d8d9f12c48","entry":"extract_patches","repo":"ewrfcas/MAE-FAR","repo_kind":"official","path":"ACR/networks/generators.py","file_url":"https://github.com/ewrfcas/MAE-FAR/blob/HEAD/ACR/networks/generators.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"d1f2e1d8d9f12c48"}},{"code_sha256_prefix":"48b341edab1e2386","entry":"ACRModel","repo":"ewrfcas/MAE-FAR","repo_kind":"official","path":"ACR/networks/generators.py","file_url":"https://github.com/ewrfcas/MAE-FAR/blob/HEAD/ACR/networks/generators.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"48b341edab1e2386"}},{"code_sha256_prefix":"81c5a1defd65ad90","entry":"FFC","repo":"ewrfcas/MAE-FAR","repo_kind":"official","path":"ACR/networks/generators.py","file_url":"https://github.com/ewrfcas/MAE-FAR/blob/HEAD/ACR/networks/generators.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"81c5a1defd65ad90"}},{"code_sha256_prefix":"78e62730a8497f98","entry":"FFCGenerator","repo":"ewrfcas/MAE-FAR","repo_kind":"official","path":"ACR/networks/generators.py","file_url":"https://github.com/ewrfcas/MAE-FAR/blob/HEAD/ACR/networks/generators.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"78e62730a8497f98"}},{"code_sha256_prefix":"b13f15c9ee45a4cd","entry":"FFCResnetBlock","repo":"ewrfcas/MAE-FAR","repo_kind":"official","path":"ACR/networks/generators.py","file_url":"https://github.com/ewrfcas/MAE-FAR/blob/HEAD/ACR/networks/generators.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"b13f15c9ee45a4cd"}},{"code_sha256_prefix":"89fdcce3884bdbb0","entry":"FFC_BN_ACT","repo":"ewrfcas/MAE-FAR","repo_kind":"official","path":"ACR/networks/generators.py","file_url":"https://github.com/ewrfcas/MAE-FAR/blob/HEAD/ACR/networks/generators.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"89fdcce3884bdbb0"}},{"code_sha256_prefix":"0f59a3bfb61b0021","entry":"FourierUnit","repo":"ewrfcas/MAE-FAR","repo_kind":"official","path":"ACR/networks/generators.py","file_url":"https://github.com/ewrfcas/MAE-FAR/blob/HEAD/ACR/networks/generators.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"0f59a3bfb61b0021"}},{"code_sha256_prefix":"e3dd50799f4e53e2","entry":"GCEncoder","repo":"ewrfcas/MAE-FAR","repo_kind":"official","path":"ACR/networks/generators.py","file_url":"https://github.com/ewrfcas/MAE-FAR/blob/HEAD/ACR/networks/generators.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"e3dd50799f4e53e2"}},{"code_sha256_prefix":"374a759dd4a97d62","entry":"GroupConvAttention","repo":"ewrfcas/MAE-FAR","repo_kind":"official","path":"ACR/networks/generators.py","file_url":"https://github.com/ewrfcas/MAE-FAR/blob/HEAD/ACR/networks/generators.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"374a759dd4a97d62"}},{"code_sha256_prefix":"c68c25adc7df2dd1","entry":"SpectralTransform","repo":"ewrfcas/MAE-FAR","repo_kind":"official","path":"ACR/networks/generators.py","file_url":"https://github.com/ewrfcas/MAE-FAR/blob/HEAD/ACR/networks/generators.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"c68c25adc7df2dd1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}