{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/image-generation-from-sketch-constraint-using","title":"Image Generation from Sketch Constraint Using Contextual GAN","arxiv_id":"1711.08972","date":"2017-11-24","proceeding":"ECCV 2018 9","authors":["Yongyi Lu","Shangzhe Wu","Yu-Wing Tai","Chi-Keung Tang"],"abstract":"In this paper we investigate image generation guided by hand sketch. When the\ninput sketch is badly drawn, the output of common image-to-image translation\nfollows the input edges due to the hard condition imposed by the translation\nprocess. Instead, we propose to use sketch as weak constraint, where the output\nedges do not necessarily follow the input edges. We address this problem using\na novel joint image completion approach, where the sketch provides the image\ncontext for completing, or generating the output image. We train a generated\nadversarial network, i.e, contextual GAN to learn the joint distribution of\nsketch and the corresponding image by using joint images. Our contextual GAN\nhas several advantages. First, the simple joint image representation allows for\nsimple and effective learning of joint distribution in the same image-sketch\nspace, which avoids complicated issues in cross-domain learning. Second, while\nthe output is related to its input overall, the generated features exhibit more\nfreedom in appearance and do not strictly align with the input features as\nprevious conditional GANs do. Third, from the joint image's point of view,\nimage and sketch are of no difference, thus exactly the same deep joint image\ncompletion network can be used for image-to-sketch generation. Experiments\nevaluated on three different datasets show that our contextual GAN can generate\nmore realistic images than state-of-the-art conditional GANs on challenging\ninputs and generalize well on common categories.","url_abs":"http://arxiv.org/abs/1711.08972v2","url_pdf":"http://arxiv.org/pdf/1711.08972v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"image-generation-from-sketch-constraint-using","repo_url":"https://github.com/elliottwu/stext2image","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"image-to-image-translation","task_name":"Image-to-Image Translation"},{"task_slug":"translation","task_name":"Translation"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1711.08972","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}