{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-the-evaluation-of-conditional-gans","title":"On the Evaluation of Conditional GANs","arxiv_id":"1907.08175","date":"2019-07-11","proceeding":null,"authors":["Terrance DeVries","Adriana Romero","Luis Pineda","Graham W. Taylor","Michal Drozdzal"],"abstract":"Conditional Generative Adversarial Networks (cGANs) are finding increasingly widespread use in many application domains. Despite outstanding progress, quantitative evaluation of such models often involves multiple distinct metrics to assess different desirable properties, such as image quality, conditional consistency, and intra-conditioning diversity. In this setting, model benchmarking becomes a challenge, as each metric may indicate a different \"best\" model. In this paper, we propose the Frechet Joint Distance (FJD), which is defined as the Frechet distance between joint distributions of images and conditioning, allowing it to implicitly capture the aforementioned properties in a single metric. We conduct proof-of-concept experiments on a controllable synthetic dataset, which consistently highlight the benefits of FJD when compared to currently established metrics. Moreover, we use the newly introduced metric to compare existing cGAN-based models for a variety of conditioning modalities (e.g. class labels, object masks, bounding boxes, images, and text captions). We show that FJD can be used as a promising single metric for cGAN benchmarking and model selection. Code can be found at https://github.com/facebookresearch/fjd.","url_abs":"https://arxiv.org/abs/1907.08175v3","url_pdf":"https://arxiv.org/pdf/1907.08175v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-the-evaluation-of-conditional-gans","repo_url":"https://github.com/facebookresearch/fjd","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"model-selection","task_name":"Model Selection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1907.08175","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1907.08175"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/facebookresearch/fjd","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":9},"by_repo_kind":{"official":{"samples":9,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"c8399c7850e0aaa2","entry":"batch_embed","repo":"facebookresearch/fjd","repo_kind":"official","path":"dsprite_experiments.py","file_url":"https://github.com/facebookresearch/fjd/blob/HEAD/dsprite_experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"c8399c7850e0aaa2"}},{"code_sha256_prefix":"41261466bf7d062f","entry":"calculate_alpha","repo":"facebookresearch/fjd","repo_kind":"official","path":"fjd_metric.py","file_url":"https://github.com/facebookresearch/fjd/blob/HEAD/fjd_metric.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"41261466bf7d062f"}},{"code_sha256_prefix":"e4c005224d71d895","entry":"calculate_fd","repo":"facebookresearch/fjd","repo_kind":"official","path":"fjd_metric.py","file_url":"https://github.com/facebookresearch/fjd/blob/HEAD/fjd_metric.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"e4c005224d71d895"}},{"code_sha256_prefix":"73243ef1bae6c55c","entry":"center_crop","repo":"facebookresearch/fjd","repo_kind":"official","path":"datasets/dSprite_textures/dsprite_utils.py","file_url":"https://github.com/facebookresearch/fjd/blob/HEAD/datasets/dSprite_textures/dsprite_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"73243ef1bae6c55c"}},{"code_sha256_prefix":"bc19a70ede599d47","entry":"get_data_from_latents","repo":"facebookresearch/fjd","repo_kind":"official","path":"dsprite_experiments.py","file_url":"https://github.com/facebookresearch/fjd/blob/HEAD/dsprite_experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"bc19a70ede599d47"}},{"code_sha256_prefix":"6824cab900935ac1","entry":"get_embedding_statistics","repo":"facebookresearch/fjd","repo_kind":"official","path":"fjd_metric.py","file_url":"https://github.com/facebookresearch/fjd/blob/HEAD/fjd_metric.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"6824cab900935ac1"}},{"code_sha256_prefix":"95aa37b607a956ae","entry":"rescale","repo":"facebookresearch/fjd","repo_kind":"official","path":"datasets/dSprite_textures/dsprite_utils.py","file_url":"https://github.com/facebookresearch/fjd/blob/HEAD/datasets/dSprite_textures/dsprite_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"95aa37b607a956ae"}},{"code_sha256_prefix":"4f6056b2ba5bb865","entry":"resize","repo":"facebookresearch/fjd","repo_kind":"official","path":"datasets/dSprite_textures/dsprite_utils.py","file_url":"https://github.com/facebookresearch/fjd/blob/HEAD/datasets/dSprite_textures/dsprite_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"4f6056b2ba5bb865"}},{"code_sha256_prefix":"9c613491aa7d5573","entry":"swap_labels","repo":"facebookresearch/fjd","repo_kind":"official","path":"dsprite_experiments.py","file_url":"https://github.com/facebookresearch/fjd/blob/HEAD/dsprite_experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"9c613491aa7d5573"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}