{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/figments-and-misalignments-a-framework-for","title":"VERITE: A Robust Benchmark for Multimodal Misinformation Detection Accounting for Unimodal Bias","arxiv_id":"2304.14133","date":"2023-04-27","proceeding":null,"authors":["Stefanos-Iordanis Papadopoulos","Christos Koutlis","Symeon Papadopoulos","Panagiotis C. Petrantonakis"],"abstract":"Multimedia content has become ubiquitous on social media platforms, leading to the rise of multimodal misinformation (MM) and the urgent need for effective strategies to detect and prevent its spread. In recent years, the challenge of multimodal misinformation detection (MMD) has garnered significant attention by researchers and has mainly involved the creation of annotated, weakly annotated, or synthetically generated training datasets, along with the development of various deep learning MMD models. However, the problem of unimodal bias has been overlooked, where specific patterns and biases in MMD benchmarks can result in biased or unimodal models outperforming their multimodal counterparts on an inherently multimodal task; making it difficult to assess progress. In this study, we systematically investigate and identify the presence of unimodal bias in widely-used MMD benchmarks, namely VMU-Twitter and COSMOS. To address this issue, we introduce the \"VERification of Image-TExt pairs\" (VERITE) benchmark for MMD which incorporates real-world data, excludes \"asymmetric multimodal misinformation\" and utilizes \"modality balancing\". We conduct an extensive comparative study with a Transformer-based architecture that shows the ability of VERITE to effectively address unimodal bias, rendering it a robust evaluation framework for MMD. Furthermore, we introduce a new method -- termed Crossmodal HArd Synthetic MisAlignment (CHASMA) -- for generating realistic synthetic training data that preserve crossmodal relations between legitimate images and false human-written captions. By leveraging CHASMA in the training process, we observe consistent and notable improvements in predictive performance on VERITE; with a 9.2% increase in accuracy. We release our code at: https://github.com/stevejpapad/image-text-verification","url_abs":"https://arxiv.org/abs/2304.14133v3","url_pdf":"https://arxiv.org/pdf/2304.14133v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"figments-and-misalignments-a-framework-for","repo_url":"https://github.com/stevejpapad/figments-and-misalignments","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"figments-and-misalignments-a-framework-for","repo_url":"https://github.com/stevejpapad/image-text-verification","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"misinformation","task_name":"Misinformation"}],"methods":[],"datasets_introduced":[{"slug":"verite","name":"VERITE","full_name":"VERification of Image-TExt pairs"}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2304.14133","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2304.14133"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/stevejpapad/figments-and-misalignments","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/stevejpapad/image-text-verification","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":5,"ran":1,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ebda1f758f7fcb92","entry":"find_images","repo":"stevejpapad/image-text-verification","repo_kind":"official","path":"extract_features.py","file_url":"https://github.com/stevejpapad/image-text-verification/blob/HEAD/extract_features.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ebda1f758f7fcb92"}},{"code_sha256_prefix":"2d6c4da1031026ab","entry":"binary_acc","repo":"stevejpapad/figments-and-misalignments","repo_kind":"official","path":"utils.py","file_url":"https://github.com/stevejpapad/figments-and-misalignments/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2d6c4da1031026ab"}},{"code_sha256_prefix":"377fa9464398a7a7","entry":"load_dataset","repo":"stevejpapad/image-text-verification","repo_kind":"official","path":"extract_features.py","file_url":"https://github.com/stevejpapad/image-text-verification/blob/HEAD/extract_features.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"377fa9464398a7a7"}},{"code_sha256_prefix":"ee3f6929dd21fa19","entry":"prepare_dataloader","repo":"stevejpapad/figments-and-misalignments","repo_kind":"official","path":"utils.py","file_url":"https://github.com/stevejpapad/figments-and-misalignments/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ee3f6929dd21fa19"}},{"code_sha256_prefix":"9adac86ab6a4ec5b","entry":"topsis","repo":"stevejpapad/figments-and-misalignments","repo_kind":"official","path":"utils.py","file_url":"https://github.com/stevejpapad/figments-and-misalignments/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9adac86ab6a4ec5b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}