{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-stable-and-faithful-inpainting","title":"Towards Enhanced Image Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency","arxiv_id":"2312.04831","date":"2023-12-08","proceeding":"CVPR 2025 1","authors":["Yikai Wang","Chenjie Cao","Junqiu Yu","Ke Fan","xiangyang xue","Yanwei Fu"],"abstract":"Recent advances in image inpainting increasingly use generative models to handle large irregular masks. However, these models can create unrealistic inpainted images due to two main issues: (1) Unwanted object insertion: Even with unmasked areas as context, generative models may still generate arbitrary objects in the masked region that don't align with the rest of the image. (2) Color inconsistency: Inpainted regions often have color shifts that causes a smeared appearance, reducing image quality. Retraining the generative model could help solve these issues, but it's costly since state-of-the-art latent-based diffusion and rectified flow models require a three-stage training process: training a VAE, training a generative U-Net or transformer, and fine-tuning for inpainting. Instead, this paper proposes a post-processing approach, dubbed as ASUKA (Aligned Stable inpainting with UnKnown Areas prior), to improve inpainting models. To address unwanted object insertion, we leverage a Masked Auto-Encoder (MAE) for reconstruction-based priors. This mitigates object hallucination while maintaining the model's generation capabilities. To address color inconsistency, we propose a specialized VAE decoder that treats latent-to-image decoding as a local harmonization task, significantly reducing color shifts for color-consistent inpainting. We validate ASUKA on SD 1.5 and FLUX inpainting variants with Places2 and MISATO, our proposed diverse collection of datasets. Results show that ASUKA mitigates object hallucination and improves color consistency over standard diffusion and rectified flow models and other inpainting methods.","url_abs":"https://arxiv.org/abs/2312.04831v3","url_pdf":"https://arxiv.org/pdf/2312.04831v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-stable-and-faithful-inpainting","repo_url":"https://github.com/yikai-wang/asuka-misato","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"hallucination","task_name":"Hallucination"},{"task_slug":"image-inpainting","task_name":"Image Inpainting"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-hallucination","task_name":"Object Hallucination"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"pixel-prediction","method_name":"Inpainting"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-inpainting-on-places2-1","task":"Image Inpainting","dataset":"Places2","model":"ASUKA","rank_in_archive_order":1,"of":14,"metrics":{"FID":"1.230","LPIPS":"0.183","P-IDS":"28.70","U-IDS":"41.30"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.04831","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.04831"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yikai-wang/asuka-misato","reach":{"status":"ok"}}],"summary":{"ran":10,"ran_violates":2,"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":17,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":17,"samples":[{"code_sha256_prefix":"e978907f8cb52268","entry":"add_border_and_mask","repo":"yikai-wang/asuka-misato","repo_kind":"official","path":"asuka-flux/demo_gradio.py","file_url":"https://github.com/yikai-wang/asuka-misato/blob/HEAD/asuka-flux/demo_gradio.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e978907f8cb52268"}},{"code_sha256_prefix":"4d09e9bc9c8e653f","entry":"cal","repo":"yikai-wang/asuka-misato","repo_kind":"official","path":"asuka-flux/eval/cal_ids.py","file_url":"https://github.com/yikai-wang/asuka-misato/blob/HEAD/asuka-flux/eval/cal_ids.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4d09e9bc9c8e653f"}},{"code_sha256_prefix":"c5b6bb489aa439ac","entry":"calculate_gradient_gt_mean","repo":"yikai-wang/asuka-misato","repo_kind":"official","path":"asuka-flux/eval/cal_grad.py","file_url":"https://github.com/yikai-wang/asuka-misato/blob/HEAD/asuka-flux/eval/cal_grad.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c5b6bb489aa439ac"}},{"code_sha256_prefix":"424012cb37b31172","entry":"default","repo":"yikai-wang/asuka-misato","repo_kind":"official","path":"asuka-flux/ldm/modules/attention.py","file_url":"https://github.com/yikai-wang/asuka-misato/blob/HEAD/asuka-flux/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"424012cb37b31172"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"yikai-wang/asuka-misato","repo_kind":"official","path":"asuka-flux/ldm/modules/attention.py","file_url":"https://github.com/yikai-wang/asuka-misato/blob/HEAD/asuka-flux/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"d81f4c3b50cd3c53","entry":"find_res_dirs","repo":"yikai-wang/asuka-misato","repo_kind":"official","path":"asuka-flux/eval/cal_sum.py","file_url":"https://github.com/yikai-wang/asuka-misato/blob/HEAD/asuka-flux/eval/cal_sum.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d81f4c3b50cd3c53"}},{"code_sha256_prefix":"112f734db631ff3a","entry":"get_result_image","repo":"yikai-wang/asuka-misato","repo_kind":"official","path":"asuka-flux/eval/cal_grad.py","file_url":"https://github.com/yikai-wang/asuka-misato/blob/HEAD/asuka-flux/eval/cal_grad.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"112f734db631ff3a"}},{"code_sha256_prefix":"b1368330cf0f5642","entry":"isimage","repo":"yikai-wang/asuka-misato","repo_kind":"official","path":"asuka-flux/ldm/util.py","file_url":"https://github.com/yikai-wang/asuka-misato/blob/HEAD/asuka-flux/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b1368330cf0f5642"}},{"code_sha256_prefix":"d72762b700feee6f","entry":"ismap","repo":"yikai-wang/asuka-misato","repo_kind":"official","path":"asuka-flux/ldm/util.py","file_url":"https://github.com/yikai-wang/asuka-misato/blob/HEAD/asuka-flux/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d72762b700feee6f"}},{"code_sha256_prefix":"a76fc7fb6958f97c","entry":"make_nograd_func","repo":"yikai-wang/asuka-misato","repo_kind":"official","path":"asuka-flux/ldm/modules/utils.py","file_url":"https://github.com/yikai-wang/asuka-misato/blob/HEAD/asuka-flux/ldm/modules/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a76fc7fb6958f97c"}},{"code_sha256_prefix":"b62bec09ceeaca79","entry":"make_recursive_func","repo":"yikai-wang/asuka-misato","repo_kind":"official","path":"asuka-flux/ldm/modules/utils.py","file_url":"https://github.com/yikai-wang/asuka-misato/blob/HEAD/asuka-flux/ldm/modules/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b62bec09ceeaca79"}},{"code_sha256_prefix":"f32aecfb3ca6c2af","entry":"prepare_data","repo":"yikai-wang/asuka-misato","repo_kind":"official","path":"asuka-flux/datas/misato_dataset.py","file_url":"https://github.com/yikai-wang/asuka-misato/blob/HEAD/asuka-flux/datas/misato_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f32aecfb3ca6c2af"}},{"code_sha256_prefix":"9a299fe5ae09e407","entry":"uniq","repo":"yikai-wang/asuka-misato","repo_kind":"official","path":"asuka-flux/ldm/modules/attention.py","file_url":"https://github.com/yikai-wang/asuka-misato/blob/HEAD/asuka-flux/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9a299fe5ae09e407"}},{"code_sha256_prefix":"8abb8e92c3b3495e","entry":"calculate_ssim","repo":"yikai-wang/asuka-misato","repo_kind":"official","path":"asuka-flux/eval/cal_psnr_lpips_ssim.py","file_url":"https://github.com/yikai-wang/asuka-misato/blob/HEAD/asuka-flux/eval/cal_psnr_lpips_ssim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8abb8e92c3b3495e"}},{"code_sha256_prefix":"37e2b5e4925fedc7","entry":"log_txt_as_img","repo":"yikai-wang/asuka-misato","repo_kind":"official","path":"asuka-flux/ldm/util.py","file_url":"https://github.com/yikai-wang/asuka-misato/blob/HEAD/asuka-flux/ldm/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"37e2b5e4925fedc7"}},{"code_sha256_prefix":"5d289ebf7d9bcada","entry":"process_batch","repo":"yikai-wang/asuka-misato","repo_kind":"official","path":"asuka-flux/eval/cal_psnr_lpips_ssim.py","file_url":"https://github.com/yikai-wang/asuka-misato/blob/HEAD/asuka-flux/eval/cal_psnr_lpips_ssim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5d289ebf7d9bcada"}},{"code_sha256_prefix":"8e28f96374f1b285","entry":"tensor2float","repo":"yikai-wang/asuka-misato","repo_kind":"official","path":"asuka-flux/ldm/modules/utils.py","file_url":"https://github.com/yikai-wang/asuka-misato/blob/HEAD/asuka-flux/ldm/modules/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8e28f96374f1b285"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}