{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sem-gan-semantically-consistent-image-to","title":"Sem-GAN: Semantically-Consistent Image-to-Image Translation","arxiv_id":"1807.04409","date":"2018-07-12","proceeding":null,"authors":["Anoop Cherian","Alan Sullivan"],"abstract":"Unpaired image-to-image translation is the problem of mapping an image in the\nsource domain to one in the target domain, without requiring corresponding\nimage pairs. To ensure the translated images are realistically plausible,\nrecent works, such as Cycle-GAN, demands this mapping to be invertible. While,\nthis requirement demonstrates promising results when the domains are unimodal,\nits performance is unpredictable in a multi-modal scenario such as in an image\nsegmentation task. This is because, invertibility does not necessarily enforce\nsemantic correctness. To this end, we present a semantically-consistent GAN\nframework, dubbed Sem-GAN, in which the semantics are defined by the class\nidentities of image segments in the source domain as produced by a semantic\nsegmentation algorithm. Our proposed framework includes consistency constraints\non the translation task that, together with the GAN loss and the\ncycle-constraints, enforces that the images when translated will inherit the\nappearances of the target domain, while (approximately) maintaining their\nidentities from the source domain. We present experiments on several\nimage-to-image translation tasks and demonstrate that Sem-GAN improves the\nquality of the translated images significantly, sometimes by more than 20% on\nthe FCN score. Further, we show that semantic segmentation models, trained with\nsynthetic images translated via Sem-GAN, leads to significantly better\nsegmentation results than other variants.","url_abs":"http://arxiv.org/abs/1807.04409v1","url_pdf":"http://arxiv.org/pdf/1807.04409v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sem-gan-semantically-consistent-image-to","repo_url":"https://github.com/mengweiren/segmentation-renormalized-harmonization","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"image-to-image-translation","task_name":"Image-to-Image Translation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"translation","task_name":"Translation"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"fcn","method_name":"FCN"},{"method_slug":"max-pooling","method_name":"Max Pooling"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1807.04409","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}