{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/forgery-aware-adaptive-transformer-for","title":"Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detection","arxiv_id":"2312.16649","date":"2023-12-27","proceeding":"CVPR 2024 1","authors":["Huan Liu","Zichang Tan","Chuangchuang Tan","Yunchao Wei","Yao Zhao","Jingdong Wang"],"abstract":"In this paper, we study the problem of generalizable synthetic image detection, aiming to detect forgery images from diverse generative methods, e.g., GANs and diffusion models. Cutting-edge solutions start to explore the benefits of pre-trained models, and mainly follow the fixed paradigm of solely training an attached classifier, e.g., combining frozen CLIP-ViT with a learnable linear layer in UniFD. However, our analysis shows that such a fixed paradigm is prone to yield detectors with insufficient learning regarding forgery representations. We attribute the key challenge to the lack of forgery adaptation, and present a novel forgery-aware adaptive transformer approach, namely FatFormer. Based on the pre-trained vision-language spaces of CLIP, FatFormer introduces two core designs for the adaption to build generalized forgery representations. First, motivated by the fact that both image and frequency analysis are essential for synthetic image detection, we develop a forgery-aware adapter to adapt image features to discern and integrate local forgery traces within image and frequency domains. Second, we find that considering the contrastive objectives between adapted image features and text prompt embeddings, a previously overlooked aspect, results in a nontrivial generalization improvement. Accordingly, we introduce language-guided alignment to supervise the forgery adaptation with image and text prompts in FatFormer. Experiments show that, by coupling these two designs, our approach tuned on 4-class ProGAN data attains a remarkable detection performance, achieving an average of 98% accuracy to unseen GANs, and surprisingly generalizes to unseen diffusion models with 95% accuracy.","url_abs":"https://arxiv.org/abs/2312.16649v1","url_pdf":"https://arxiv.org/pdf/2312.16649v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"forgery-aware-adaptive-transformer-for","repo_url":"https://github.com/Michel-liu/FatFormer","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"forgery-aware-adaptive-transformer-for","repo_url":"https://github.com/scu-zjz/forensichub","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"CC-BY-4.0"}}],"tasks":[{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"synthetic-image-detection","task_name":"Synthetic Image Detection"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"adapter","method_name":"Adapter"},{"method_slug":"clip","method_name":"CLIP"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"local-response-normalization","method_name":"Local Response Normalization"},{"method_slug":"progan","method_name":"ProGAN"},{"method_slug":"wgan-gp-loss","method_name":"WGAN-GP Loss"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2312.16649","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.16649"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Michel-liu/FatFormer","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/scu-zjz/forensichub","reach":{"status":"ok","spdx":"CC-BY-4.0"}}],"summary":{"ran":3,"ran_draft_wrong":3,"unverified":1},"by_repo_kind":{"official":{"samples":7,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"aa6da43d05cfaf65","entry":"all_gather","repo":"Michel-liu/FatFormer","repo_kind":"official","path":"utils/misc.py","file_url":"https://github.com/Michel-liu/FatFormer/blob/HEAD/utils/misc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aa6da43d05cfaf65"}},{"code_sha256_prefix":"98f385d847636a3e","entry":"basic_clean","repo":"Michel-liu/FatFormer","repo_kind":"official","path":"models/clip/simple_tokenizer.py","file_url":"https://github.com/Michel-liu/FatFormer/blob/HEAD/models/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"98f385d847636a3e"}},{"code_sha256_prefix":"5b025be14d766797","entry":"collate_fn","repo":"Michel-liu/FatFormer","repo_kind":"official","path":"utils/misc.py","file_url":"https://github.com/Michel-liu/FatFormer/blob/HEAD/utils/misc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5b025be14d766797"}},{"code_sha256_prefix":"d919ae32e5e4e616","entry":"get_pairs","repo":"Michel-liu/FatFormer","repo_kind":"official","path":"models/clip/simple_tokenizer.py","file_url":"https://github.com/Michel-liu/FatFormer/blob/HEAD/models/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d919ae32e5e4e616"}},{"code_sha256_prefix":"05992505df9ef0cd","entry":"reduce_dict","repo":"Michel-liu/FatFormer","repo_kind":"official","path":"utils/misc.py","file_url":"https://github.com/Michel-liu/FatFormer/blob/HEAD/utils/misc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"05992505df9ef0cd"}},{"code_sha256_prefix":"9542161e9640b858","entry":"whitespace_clean","repo":"Michel-liu/FatFormer","repo_kind":"official","path":"models/clip/simple_tokenizer.py","file_url":"https://github.com/Michel-liu/FatFormer/blob/HEAD/models/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9542161e9640b858"}},{"code_sha256_prefix":"b06286bdf5f9c373","entry":"load","repo":"Michel-liu/FatFormer","repo_kind":"official","path":"models/clip/clip.py","file_url":"https://github.com/Michel-liu/FatFormer/blob/HEAD/models/clip/clip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b06286bdf5f9c373"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}