{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unleashing-the-potential-of-consistency-1","title":"Unleashing the Potential of Consistency Learning for Detecting and Grounding Multi-Modal Media Manipulation","arxiv_id":"2506.05890","date":"2025-06-06","proceeding":"CVPR 2025 1","authors":["Yiheng Li","Yang Yang","Zichang Tan","Huan Liu","Weihua Chen","Xu Zhou","Zhen Lei"],"abstract":"To tackle the threat of fake news, the task of detecting and grounding multi-modal media manipulation DGM4 has received increasing attention. However, most state-of-the-art methods fail to explore the fine-grained consistency within local content, usually resulting in an inadequate perception of detailed forgery and unreliable results. In this paper, we propose a novel approach named Contextual-Semantic Consistency Learning (CSCL) to enhance the fine-grained perception ability of forgery for DGM4. Two branches for image and text modalities are established, each of which contains two cascaded decoders, i.e., Contextual Consistency Decoder (CCD) and Semantic Consistency Decoder (SCD), to capture within-modality contextual consistency and across-modality semantic consistency, respectively. Both CCD and SCD adhere to the same criteria for capturing fine-grained forgery details. To be specific, each module first constructs consistency features by leveraging additional supervision from the heterogeneous information of each token pair. Then, the forgery-aware reasoning or aggregating is adopted to deeply seek forgery cues based on the consistency features. Extensive experiments on DGM4 datasets prove that CSCL achieves new state-of-the-art performance, especially for the results of grounding manipulated content. Codes and weights are avaliable at https://github.com/liyih/CSCL.","url_abs":"https://arxiv.org/abs/2506.05890v1","url_pdf":"https://arxiv.org/pdf/2506.05890v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unleashing-the-potential-of-consistency-1","repo_url":"https://github.com/liyih/cscl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2506.05890","atlas_url":"https://app.syntology.ai/?focus=2506.05890","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.05890"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/liyih/CSCL","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":6,"ran_fixture":3,"ran_honours":2,"ran_draft_wrong":1,"ran_violates":1,"unverified":11},"by_repo_kind":{"official":{"samples":24,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"24bfe03a8fb0e475","entry":"adapt_position_encoding","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/models/METER/clip_model.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/models/METER/clip_model.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"24bfe03a8fb0e475"}},{"code_sha256_prefix":"839a33f049497ac2","entry":"all_gather","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/models/METER/dist_utils.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/models/METER/dist_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"839a33f049497ac2"}},{"code_sha256_prefix":"78ca1debd9c4ed87","entry":"autocontrast_func","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/dataset/randaugment.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/dataset/randaugment.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"78ca1debd9c4ed87"}},{"code_sha256_prefix":"e0a06ded5d4f6c3c","entry":"box_cxcywh_to_xyxy","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/models/box_ops.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/models/box_ops.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e0a06ded5d4f6c3c"}},{"code_sha256_prefix":"7f5165b3c0380ff2","entry":"box_iou","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/models/box_ops.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/models/box_ops.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f5165b3c0380ff2"}},{"code_sha256_prefix":"031b02fb990284ba","entry":"box_xyxy_to_cxcywh","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/models/box_ops.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/models/box_ops.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"031b02fb990284ba"}},{"code_sha256_prefix":"aa81d03f07735ab8","entry":"compute_acc","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/utils.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/utils.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa81d03f07735ab8"}},{"code_sha256_prefix":"ad12a494674d23fb","entry":"compute_n_params","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/utils.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ad12a494674d23fb"}},{"code_sha256_prefix":"4c53169f63a1fcef","entry":"equalize_func","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/dataset/randaugment.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/dataset/randaugment.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4c53169f63a1fcef"}},{"code_sha256_prefix":"cc8ab2b168b5373d","entry":"gather","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/models/METER/dist_utils.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/models/METER/dist_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cc8ab2b168b5373d"}},{"code_sha256_prefix":"1dd2ab205689d45b","entry":"identity_func","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/dataset/randaugment.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/dataset/randaugment.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1dd2ab205689d45b"}},{"code_sha256_prefix":"01849cd80343b88a","entry":"pre_question","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/dataset/utils.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/dataset/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"01849cd80343b88a"}},{"code_sha256_prefix":"53838c15d969eb79","entry":"reduce_dict","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/models/METER/dist_utils.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/models/METER/dist_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"53838c15d969eb79"}},{"code_sha256_prefix":"342d33b0c5ebee8f","entry":"add_multiline_text","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/utils.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"342d33b0c5ebee8f"}},{"code_sha256_prefix":"84631e1ece975f6b","entry":"build_model","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/models/METER/clip_model.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/models/METER/clip_model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"84631e1ece975f6b"}},{"code_sha256_prefix":"22c7fff0fb7d9049","entry":"calculate_patch_labels","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/models/consist_modeling.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/models/consist_modeling.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"22c7fff0fb7d9049"}},{"code_sha256_prefix":"e67c2a64bd9a69e2","entry":"coords_2d","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/models/CSCL.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/models/CSCL.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e67c2a64bd9a69e2"}},{"code_sha256_prefix":"c04888604cde46ea","entry":"get_sscore_label","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/models/consist_modeling.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/models/consist_modeling.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c04888604cde46ea"}},{"code_sha256_prefix":"a5464e89a64cec02","entry":"get_sscore_label_text","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/models/consist_modeling.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/models/consist_modeling.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a5464e89a64cec02"}},{"code_sha256_prefix":"efbb0f3903b5f779","entry":"load_tf_weights_in_bert","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/models/METER/bert_model.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/models/METER/bert_model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"efbb0f3903b5f779"}},{"code_sha256_prefix":"fea8a4547c819eec","entry":"pos2posemb2d","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/models/CSCL.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/models/CSCL.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fea8a4547c819eec"}},{"code_sha256_prefix":"8a57c3e7d1cf36c7","entry":"pre_caption","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/dataset/utils.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/dataset/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8a57c3e7d1cf36c7"}},{"code_sha256_prefix":"9f85ec8b64bbcaac","entry":"score2posemb1d","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/models/CSCL.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/models/CSCL.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f85ec8b64bbcaac"}},{"code_sha256_prefix":"53e282c1c632bfa7","entry":"vqa_eval","repo":"liyih/CSCL","repo_kind":"official","path":"code/MultiModal-DeepFake-main/dataset/utils.py","file_url":"https://github.com/liyih/CSCL/blob/HEAD/code/MultiModal-DeepFake-main/dataset/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"53e282c1c632bfa7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}