{"url":"/method/ian","slug":"ian","name":"IAN","full_name":"Introspective Adversarial Network","full_name_withheld":false,"description_markdown":"The **Introspective Adversarial Network (IAN)** is a hybridization of [GANs](https://paperswithcode.com/method/gan) and [VAEs](https://paperswithcode.com/method/vae) that leverages the power of the adversarial objective while maintaining the VAE’s efficient inference mechanism. It uses the discriminator of the GAN, $D$, as a feature extractor for an inference subnetwork, $E$, which is implemented as a fully-connected layer on top of the final convolutional layer of the discriminator. We infer latent values $Z \\sim E\\left(X\\right) = q\\left(Z\\mid{X}\\right)$ for reconstruction and sample random values $Z \\sim p\\left(Z\\right)$ from a standard normal for random image generation using the generator network, $G$.\r\n\r\nThree distinct loss functions are used:\r\n\r\n- $\\mathcal{L}\\_{img}$, the L1 pixel-wise reconstruction loss, which is preferred to the L2 reconstruction loss for its higher average gradient.\r\n- $\\mathcal{L\\_{feature}}$, the feature-wise reconstruction loss, evaluated as the L2 difference between the original and reconstruction in the space of the hidden layers of the discriminator.\r\n- $\\mathcal{L}\\_{adv}$, the ternary adversarial loss, a modification of the adversarial loss that forces the discriminator to label a sample as real, generated, or reconstructed (as opposed to a binary\r\nreal vs. generated label).\r\n\r\nIncluding the VAE’s KL divergence between the inferred latents $E\\left(X\\right)$ and the prior $p\\left(Z\\right)$, the loss function for the generator and encoder network is thus:\r\n\r\n$$\\mathcal{L}\\_{E, G} = \\lambda\\_{adv}\\mathcal{L}\\_{G\\_{adv}} + \\lambda\\_{img}\\mathcal{L}\\_{img}  + \\lambda\\_{feature}\\mathcal{L}\\_{feature}  + D\\_{KL}\\left(E\\left(X\\right) || p\\left(Z\\right)\\right) $$\r\n\r\nWhere the $\\lambda$ terms weight the relative importance of each loss. We set $\\lambda\\_{img}$ to 3 and leave the other terms at 1. The discriminator is updated solely using the ternary adversarial loss. During each training step, the generator produces reconstructions $G\\left(E\\left(X\\right)\\right)$ (using the standard VAE reparameterization trick) from data $X$ and random samples $G\\left(Z\\right)$, while the discriminator observes $X$ as well as the reconstructions and random samples, and both networks are simultaneously updated.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"http://arxiv.org/abs/1609.07093v3","title":"Neural Photo Editing with Introspective Adversarial Networks","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/ajbrock/Neural-Photo-Editor/blob/master/IAN.py","code_snippet_url_on_a_code_host":true,"categories":[{"area":"Computer Vision","area_id":"computer-vision","collection":"Generative Models","url":"/methods/category/generative-models","pwc_aliases":[]}],"n_papers_tagged":4,"archive_num_papers":null,"papers_newest_first":[{"paper":"/paper/cross-view-geolocalization-and-disaster","title":"Cross-View Geolocalization and Disaster Mapping with Street-View and VHR Satellite Imagery: A Case Study of Hurricane IAN","date":"2024-08-13","arxiv_id":"2408.06761","n_code_links":1,"syntology":null},{"paper":"/paper/inferior-alveolar-nerve-segmentation-in-cbct","title":"Inferior Alveolar Nerve Segmentation in CBCT images using Connectivity-Based Selective Re-training","date":"2023-08-18","arxiv_id":"2308.09298","n_code_links":1,"syntology":null},{"paper":null,"title":"Arabic aspect based sentiment analysis using bidirectional GRU based models","date":"2021-01-23","arxiv_id":"2101.10539","n_code_links":0,"syntology":null},{"paper":"/paper/neural-photo-editing-with-introspective","title":"Neural Photo Editing with Introspective Adversarial Networks","date":"2016-09-22","arxiv_id":"1609.07093","n_code_links":2,"syntology":{"ran":2,"of":2,"unverified":0,"pointer_only":0}}],"papers_shown":4,"tasks":[{"task":"/task/aspect-based-sentiment-analysis-1","name":"Aspect-Based Sentiment Analysis","papers":1},{"task":"/task/aspect-based-sentiment-analysis","name":"Aspect-Based Sentiment Analysis (ABSA)","papers":1},{"task":"/task/contrastive-learning","name":"Contrastive Learning","papers":1},{"task":"/task/disaster-response","name":"Disaster Response","papers":1},{"task":"/task/image-generation","name":"Image Generation","papers":1},{"task":"/task/sentence","name":"Sentence","papers":1},{"task":"/task/sentiment-analysis","name":"Sentiment Analysis","papers":1}],"tasks_shown":7,"n_tasks":7,"usage_by_year":[{"year":"2016","papers":1},{"year":"2021","papers":1},{"year":"2023","papers":1},{"year":"2024","papers":1}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/ian"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}