{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/improving-consistency-and-correctness-of","title":"Improving Consistency and Correctness of Sequence Inpainting using Semantically Guided Generative Adversarial Network","arxiv_id":"1711.06106","date":"2017-11-16","proceeding":null,"authors":["Avisek Lahiri","Arnav Jain","Prabir Kumar Biswas","Pabitra Mitra"],"abstract":"Contemporary benchmark methods for image inpainting are based on deep\ngenerative models and specifically leverage adversarial loss for yielding\nrealistic reconstructions. However, these models cannot be directly applied on\nimage/video sequences because of an intrinsic drawback- the reconstructions\nmight be independently realistic, but, when visualized as a sequence, often\nlacks fidelity to the original uncorrupted sequence. The fundamental reason is\nthat these methods try to find the best matching latent space representation\nnear to natural image manifold without any explicit distance based loss. In\nthis paper, we present a semantically conditioned Generative Adversarial\nNetwork (GAN) for sequence inpainting. The conditional information constrains\nthe GAN to map a latent representation to a point in image manifold respecting\nthe underlying pose and semantics of the scene. To the best of our knowledge,\nthis is the first work which simultaneously addresses consistency and\ncorrectness of generative model based inpainting. We show that our generative\nmodel learns to disentangle pose and appearance information; this independence\nis exploited by our model to generate highly consistent reconstructions. The\nconditional information also aids the generator network in GAN to produce\nsharper images compared to the original GAN formulation. This helps in\nachieving more appealing inpainting performance. Though generic, our algorithm\nwas targeted for inpainting on faces. When applied on CelebA and Youtube Faces\ndatasets, the proposed method results in a significant improvement over the\ncurrent benchmark, both in terms of quantitative evaluation (Peak Signal to\nNoise Ratio) and human visual scoring over diversified combinations of\nresolutions and deformations.","url_abs":"http://arxiv.org/abs/1711.06106v2","url_pdf":"http://arxiv.org/pdf/1711.06106v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"improving-consistency-and-correctness-of","repo_url":"https://github.com/arnavkj1995/face_inpainting","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":null,"task_name":"Generative Adversarial Network"},{"task_slug":"image-inpainting","task_name":"Image Inpainting"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1711.06106","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}