{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fakeshield-explainable-image-forgery","title":"FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models","arxiv_id":"2410.02761","date":"2024-10-03","proceeding":null,"authors":["Zhipei Xu","Xuanyu Zhang","Runyi Li","Zecheng Tang","Qing Huang","Jian Zhang"],"abstract":"The rapid development of generative AI is a double-edged sword, which not only facilitates content creation but also makes image manipulation easier and more difficult to detect. Although current image forgery detection and localization (IFDL) methods are generally effective, they tend to face two challenges: \\textbf{1)} black-box nature with unknown detection principle, \\textbf{2)} limited generalization across diverse tampering methods (e.g., Photoshop, DeepFake, AIGC-Editing). To address these issues, we propose the explainable IFDL task and design FakeShield, a multi-modal framework capable of evaluating image authenticity, generating tampered region masks, and providing a judgment basis based on pixel-level and image-level tampering clues. Additionally, we leverage GPT-4o to enhance existing IFDL datasets, creating the Multi-Modal Tamper Description dataSet (MMTD-Set) for training FakeShield's tampering analysis capabilities. Meanwhile, we incorporate a Domain Tag-guided Explainable Forgery Detection Module (DTE-FDM) and a Multi-modal Forgery Localization Module (MFLM) to address various types of tamper detection interpretation and achieve forgery localization guided by detailed textual descriptions. Extensive experiments demonstrate that FakeShield effectively detects and localizes various tampering techniques, offering an explainable and superior solution compared to previous IFDL methods.","url_abs":"https://arxiv.org/abs/2410.02761v3","url_pdf":"https://arxiv.org/pdf/2410.02761v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fakeshield-explainable-image-forgery","repo_url":"https://github.com/zhipeixu/fakeshield","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"fakeshield-explainable-image-forgery","repo_url":"https://github.com/Recognito-Vision/Linux-FaceRecognition-FaceLivenessDetection","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"face-swapping","task_name":"Face Swapping"},{"task_slug":"image-forgery-detection","task_name":"Image Forgery Detection"},{"task_slug":"image-manipulation","task_name":"Image Manipulation"},{"task_slug":"tag","task_name":"TAG"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.02761","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.02761"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Recognito-Vision/Linux-FaceRecognition-FaceLivenessDetection","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhipeixu/fakeshield","reach":null}],"summary":{"ran_violates":1,"ran_draft_wrong":1,"ran_fixture":1,"unverified":3},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"763f6fe20bd52fbb","entry":"parse_score","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"763f6fe20bd52fbb"}},{"code_sha256_prefix":"b1c0bb4ee3a78693","entry":"read_jsonl","repo":"zhipeixu/fakeshield","repo_kind":"official","path":"MFLM/cli_demo.py","file_url":"https://github.com/zhipeixu/fakeshield/blob/HEAD/MFLM/cli_demo.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b1c0bb4ee3a78693"}},{"code_sha256_prefix":"10893c4608c08075","entry":"split_to_even_chunks","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"10893c4608c08075"}},{"code_sha256_prefix":"bb35e3ac741bb2c9","entry":"get_mm_adapter_state_maybe_zero_3","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"bb35e3ac741bb2c9"}},{"code_sha256_prefix":"735025744c1ab0cf","entry":"maybe_zero_3","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"735025744c1ab0cf"}},{"code_sha256_prefix":"4bda04b47d778f27","entry":"parse_args","repo":"zhipeixu/fakeshield","repo_kind":"official","path":"MFLM/cli_demo.py","file_url":"https://github.com/zhipeixu/fakeshield/blob/HEAD/MFLM/cli_demo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4bda04b47d778f27"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}