{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cove-context-and-veracity-prediction-for-out","title":"COVE: COntext and VEracity prediction for out-of-context images","arxiv_id":"2502.01194","date":"2025-02-03","proceeding":null,"authors":["Jonathan Tonglet","Gabriel Thiem","Iryna Gurevych"],"abstract":"Images taken out of their context are the most prevalent form of multimodal misinformation. Debunking them requires (1) providing the true context of the image and (2) checking the veracity of the image's caption. However, existing automated fact-checking methods fail to tackle both objectives explicitly. In this work, we introduce COVE, a new method that predicts first the true COntext of the image and then uses it to predict the VEracity of the caption. COVE beats the SOTA context prediction model on all context items, often by more than five percentage points. It is competitive with the best veracity prediction models on synthetic data and outperforms them on real-world data, showing that it is beneficial to combine the two tasks sequentially. Finally, we conduct a human study that reveals that the predicted context is a reusable and interpretable artifact to verify new out-of-context captions for the same image. Our code and data are made available.","url_abs":"https://arxiv.org/abs/2502.01194v1","url_pdf":"https://arxiv.org/pdf/2502.01194v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cove-context-and-veracity-prediction-for-out","repo_url":"https://github.com/ukplab/naacl2025-cove","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"cove-context-and-veracity-prediction-for-out","repo_url":"https://github.com/ukplab/5pils","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"fact-checking","task_name":"Fact Checking"},{"task_slug":"misinformation","task_name":"Misinformation"}],"methods":[{"method_slug":"bilstm","method_name":"BiLSTM"},{"method_slug":"cove","method_name":"CoVe"},{"method_slug":"glove","method_name":"GloVe"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"location-based-attention","method_name":"Location-based Attention"},{"method_slug":"seq2seq","method_name":"Seq2Seq"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2502.01194","atlas_url":"https://app.syntology.ai/?focus=2502.01194","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.01194"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/UKPLab/naacl2025-cove","reach":null}],"summary":{"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"909509fae1d7fafb","entry":"create_input_and_labels","repo":"UKPLab/naacl2025-cove","repo_kind":"official","path":"src/veracity_deberta.py","file_url":"https://github.com/UKPLab/naacl2025-cove/blob/HEAD/src/veracity_deberta.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"909509fae1d7fafb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}