{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-the-robustness-of-quality-measures-for","title":"On the Robustness of Quality Measures for GANs","arxiv_id":"2201.13019","date":"2022-01-31","proceeding":null,"authors":["Motasem Alfarra","Juan C. Pérez","Anna Frühstück","Philip H. S. Torr","Peter Wonka","Bernard Ghanem"],"abstract":"This work evaluates the robustness of quality measures of generative models such as Inception Score (IS) and Fr\\'echet Inception Distance (FID). Analogous to the vulnerability of deep models against a variety of adversarial attacks, we show that such metrics can also be manipulated by additive pixel perturbations. Our experiments indicate that one can generate a distribution of images with very high scores but low perceptual quality. Conversely, one can optimize for small imperceptible perturbations that, when added to real world images, deteriorate their scores. We further extend our evaluation to generative models themselves, including the state of the art network StyleGANv2. We show the vulnerability of both the generative model and the FID against additive perturbations in the latent space. Finally, we show that the FID can be robustified by simply replacing the standard Inception with a robust Inception. We validate the effectiveness of the robustified metric through extensive experiments, showing it is more robust against manipulation.","url_abs":"https://arxiv.org/abs/2201.13019v2","url_pdf":"https://arxiv.org/pdf/2201.13019v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-the-robustness-of-quality-measures-for","repo_url":"https://github.com/motasemalfarra/r-fid-robustness-of-quality-measures-for-gans","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2201.13019","atlas_url":"https://app.syntology.ai/?focus=2201.13019","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2201.13019"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/motasemalfarra/r-fid-robustness-of-quality-measures-for-gans","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0cb865257b76d3fc","entry":"copy_pretrained_model","repo":"motasemalfarra/r-fid-robustness-of-quality-measures-for-gans","repo_kind":"official","path":"models/resnet.py","file_url":"https://github.com/motasemalfarra/r-fid-robustness-of-quality-measures-for-gans/blob/HEAD/models/resnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0cb865257b76d3fc"}},{"code_sha256_prefix":"3d012987836d459c","entry":"get_plugin","repo":"motasemalfarra/r-fid-robustness-of-quality-measures-for-gans","repo_kind":"official","path":"models/torch_utils/custom_ops.py","file_url":"https://github.com/motasemalfarra/r-fid-robustness-of-quality-measures-for-gans/blob/HEAD/models/torch_utils/custom_ops.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3d012987836d459c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}