{"url":"/method/pix2pix","slug":"pix2pix","name":"Pix2Pix","full_name":"Pix2Pix","full_name_withheld":false,"description_markdown":"**Pix2Pix** is a conditional image-to-image translation architecture that uses a conditional [GAN](https://paperswithcode.com/method/gan) objective combined with a reconstruction loss. The conditional GAN objective for observed images $x$, output images $y$ and the random noise vector $z$ is:\r\n\r\n$$ \\mathcal{L}\\_{cGAN}\\left(G, D\\right) =\\mathbb{E}\\_{x,y}\\left[\\log D\\left(x, y\\right)\\right]+\r\n\\mathbb{E}\\_{x,z}\\left[log(1 − D\\left(x, G\\left(x, z\\right)\\right)\\right] $$\r\n\r\nWe augment this with a reconstruction term:\r\n\r\n$$ \\mathcal{L}\\_{L1}\\left(G\\right) = \\mathbb{E}\\_{x,y,z}\\left[||y - G\\left(x, z\\right)||\\_{1}\\right] $$\r\n\r\nand we get the final objective as:\r\n\r\n$$ G^{*} = \\arg\\min\\_{G}\\max\\_{D}\\mathcal{L}\\_{cGAN}\\left(G, D\\right) + \\lambda\\mathcal{L}\\_{L1}\\left(G\\right) $$\r\n\r\nThe architectures employed for the generator and discriminator closely follow [DCGAN](https://paperswithcode.com/method/dcgan), with a few modifications:\r\n\r\n- Concatenated skip connections are used to \"shuttle\" low-level information between the input and output, similar to a [U-Net](https://paperswithcode.com/method/u-net).\r\n- The use of a [PatchGAN](https://paperswithcode.com/method/patchgan) discriminator that only penalizes structure at the scale of patches.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"http://arxiv.org/abs/1611.07004v3","title":"Image-to-Image Translation with Conditional Adversarial Networks","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix/blob/9e6fff7b7d5215a38be3cac074ca7087041bea0d/models/pix2pix_model.py#L6","code_snippet_url_on_a_code_host":true,"categories":[{"area":"Computer Vision","area_id":"computer-vision","collection":"Generative Models","url":"/methods/category/generative-models","pwc_aliases":[]}],"n_papers_tagged":132,"archive_num_papers":null,"papers_newest_first":[{"paper":null,"title":"Deep histological synthesis from mass spectrometry imaging for multimodal registration","date":"2025-06-05","arxiv_id":"2506.05441","n_code_links":0,"syntology":null},{"paper":null,"title":"Towards Generating Realistic Underwater Images","date":"2025-05-20","arxiv_id":"2505.14296","n_code_links":0,"syntology":null},{"paper":null,"title":"Object detection in adverse weather conditions for autonomous vehicles using Instruct Pix2Pix","date":"2025-05-13","arxiv_id":"2505.08228","n_code_links":0,"syntology":null},{"paper":null,"title":"Canny2Palm: Realistic and Controllable Palmprint Generation for Large-scale Pre-training","date":"2025-05-08","arxiv_id":"2505.04922","n_code_links":0,"syntology":null},{"paper":null,"title":"Whole-Body Image-to-Image Translation for a Virtual Scanner in a Healthcare Digital Twin","date":"2025-03-18","arxiv_id":"2503.15555","n_code_links":0,"syntology":null},{"paper":"/paper/generalizable-image-repair-for-robust-visual","title":"Generalizable Image Repair for Robust Visual Autonomous Racing","date":"2025-03-07","arxiv_id":"2503.05911","n_code_links":1,"syntology":null},{"paper":"/paper/a-physics-informed-deep-learning-model-for","title":"A Physics-Informed Deep Learning Model for MRI Brain Motion Correction","date":"2025-02-13","arxiv_id":"2502.09296","n_code_links":1,"syntology":null},{"paper":null,"title":"DGSSA: Domain generalization with structural and stylistic augmentation for retinal vessel segmentation","date":"2025-01-07","arxiv_id":"2501.03466","n_code_links":0,"syntology":null},{"paper":"/paper/mapping-the-mind-of-an-instruction-based","title":"Mapping the Mind of an Instruction-based Image Editing using SMILE","date":"2024-12-20","arxiv_id":"2412.16277","n_code_links":2,"syntology":null},{"paper":null,"title":"Ensemble Learning and 3D Pix2Pix for Comprehensive Brain Tumor Analysis in Multimodal MRI","date":"2024-12-16","arxiv_id":"2412.11849","n_code_links":0,"syntology":null},{"paper":null,"title":"Generative AI: A Pix2pix-GAN-Based Machine Learning Approach for Robust and Efficient Lung Segmentation","date":"2024-12-14","arxiv_id":"2412.10826","n_code_links":0,"syntology":null},{"paper":"/paper/utilizing-multi-step-loss-for-single-image","title":"Utilizing Multi-step Loss for Single Image Reflection Removal","date":"2024-12-11","arxiv_id":"2412.08582","n_code_links":1,"syntology":null},{"paper":null,"title":"AI-based 3-Lead to 12-Lead ECG Reconstruction: Towards Smartphone-based Public Healthcare","date":"2024-10-17","arxiv_id":"2410.13528","n_code_links":0,"syntology":null},{"paper":null,"title":"Synthesizing Proton-Density Fat Fraction and $R_2^*$ from 2-point Dixon MRI with Generative Machine Learning","date":"2024-10-15","arxiv_id":"2410.11186","n_code_links":0,"syntology":null},{"paper":null,"title":"Training a Computer Vision Model for Commercial Bakeries with Primarily Synthetic Images","date":"2024-09-30","arxiv_id":"2409.20122","n_code_links":0,"syntology":null},{"paper":null,"title":"Improving Cone-Beam CT Image Quality with Knowledge Distillation-Enhanced Diffusion Model in Imbalanced Data Settings","date":"2024-09-19","arxiv_id":"2409.12539","n_code_links":0,"syntology":null},{"paper":null,"title":"Enhanced Pix2Pix GAN for Visual Defect Removal in UAV-Captured Images","date":"2024-09-10","arxiv_id":"2409.06889","n_code_links":0,"syntology":null},{"paper":null,"title":"Simulating realistic short tandem repeat capillary electrophoretic signal using a generative adversarial network","date":"2024-08-28","arxiv_id":"2408.16169","n_code_links":0,"syntology":null},{"paper":null,"title":"Decision Support System to triage of liver trauma","date":"2024-08-04","arxiv_id":"2408.02012","n_code_links":0,"syntology":null},{"paper":null,"title":"VIMs: Virtual Immunohistochemistry Multiplex staining via Text-to-Stain Diffusion Trained on Uniplex Stains","date":"2024-07-26","arxiv_id":"2407.19113","n_code_links":0,"syntology":null},{"paper":null,"title":"McGAN: Generating Manufacturable Designs by Embedding Manufacturing Rules into Conditional Generative Adversarial Network","date":"2024-07-24","arxiv_id":"2407.16943","n_code_links":0,"syntology":null},{"paper":"/paper/novel-hybrid-integrated-pix2pix-and-wgan","title":"Novel Hybrid Integrated Pix2Pix and WGAN Model with Gradient Penalty for Binary Images Denoising","date":"2024-07-16","arxiv_id":"2407.11865","n_code_links":1,"syntology":null},{"paper":"/paper/cyclic-2-5d-perceptual-loss-for-cross-modal","title":"Cyclic 2.5D Perceptual Loss for Cross-Modal 3D Medical Image Synthesis: T1w MRI to Tau PET","date":"2024-06-18","arxiv_id":"2406.12632","n_code_links":1,"syntology":null},{"paper":"/paper/potatogans-utilizing-generative-adversarial","title":"PotatoGANs: Utilizing Generative Adversarial Networks, Instance Segmentation, and Explainable AI for Enhanced Potato Disease Identification and Classification","date":"2024-05-12","arxiv_id":"2405.07332","n_code_links":1,"syntology":null},{"paper":"/paper/detecting-and-refining-hirise-image-patches","title":"Detecting and Refining HiRISE Image Patches Obscured by Atmospheric Dust","date":"2024-05-08","arxiv_id":"2405.04722","n_code_links":1,"syntology":null},{"paper":null,"title":"Mapping New Realities: Ground Truth Image Creation with Pix2Pix Image-to-Image Translation","date":"2024-04-30","arxiv_id":"2404.19265","n_code_links":0,"syntology":null},{"paper":"/paper/toward-physics-aware-deep-learning","title":"Toward Physics-Aware Deep Learning Architectures for LiDAR Intensity Simulation","date":"2024-04-24","arxiv_id":"2404.15774","n_code_links":1,"syntology":null},{"paper":null,"title":"StainDiffuser: MultiTask Dual Diffusion Model for Virtual Staining","date":"2024-03-17","arxiv_id":"2403.11340","n_code_links":0,"syntology":null},{"paper":null,"title":"BraSyn 2023 challenge: Missing MRI synthesis and the effect of different learning objectives","date":"2024-03-12","arxiv_id":"2403.07800","n_code_links":0,"syntology":null},{"paper":null,"title":"Learning to Find Missing Video Frames with Synthetic Data Augmentation: A General Framework and Application in Generating Thermal Images Using RGB Cameras","date":"2024-02-29","arxiv_id":"2403.00196","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/translation","name":"Translation","papers":41},{"task":"/task/image-to-image-translation","name":"Image-to-Image Translation","papers":37},{"task":null,"name":"Generative Adversarial Network","papers":31},{"task":"/task/image-generation","name":"Image Generation","papers":19},{"task":"/task/ssim","name":"SSIM","papers":14},{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":12},{"task":"/task/segmentation","name":"Segmentation","papers":11},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":9},{"task":"/task/diagnostic","name":"Diagnostic","papers":7},{"task":"/task/colorization","name":"Colorization","papers":6},{"task":"/task/style-transfer","name":"Style Transfer","papers":6},{"task":"/task/autonomous-driving","name":"Autonomous Driving","papers":5},{"task":"/task/anatomy","name":"Anatomy","papers":4},{"task":"/task/decoder","name":"Decoder","papers":4},{"task":"/task/deep-learning","name":"Deep Learning","papers":4},{"task":"/task/denoising","name":"Denoising","papers":4},{"task":"/task/object","name":"Object","papers":4},{"task":"/task/object-detection","name":"Object Detection","papers":4},{"task":"/task/object-detection-1","name":"object-detection","papers":4},{"task":"/task/autonomous-vehicles","name":"Autonomous Vehicles","papers":3}],"tasks_shown":20,"n_tasks":133,"usage_by_year":[{"year":"2016","papers":1},{"year":"2017","papers":2},{"year":"2018","papers":1},{"year":"2019","papers":12},{"year":"2020","papers":14},{"year":"2021","papers":28},{"year":"2022","papers":15},{"year":"2023","papers":23},{"year":"2024","papers":28},{"year":"2025","papers":8}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/pix2pix"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}