{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/complex-claim-verification-with-evidence","title":"Complex Claim Verification with Evidence Retrieved in the Wild","arxiv_id":"2305.11859","date":"2023-05-19","proceeding":null,"authors":["Jifan Chen","Grace Kim","Aniruddh Sriram","Greg Durrett","Eunsol Choi"],"abstract":"Evidence retrieval is a core part of automatic fact-checking. Prior work makes simplifying assumptions in retrieval that depart from real-world use cases: either no access to evidence, access to evidence curated by a human fact-checker, or access to evidence available long after the claim has been made. In this work, we present the first fully automated pipeline to check real-world claims by retrieving raw evidence from the web. We restrict our retriever to only search documents available prior to the claim's making, modeling the realistic scenario where an emerging claim needs to be checked. Our pipeline includes five components: claim decomposition, raw document retrieval, fine-grained evidence retrieval, claim-focused summarization, and veracity judgment. We conduct experiments on complex political claims in the ClaimDecomp dataset and show that the aggregated evidence produced by our pipeline improves veracity judgments. Human evaluation finds the evidence summary produced by our system is reliable (it does not hallucinate information) and relevant to answering key questions about a claim, suggesting that it can assist fact-checkers even when it cannot surface a complete evidence set.","url_abs":"https://arxiv.org/abs/2305.11859v2","url_pdf":"https://arxiv.org/pdf/2305.11859v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"complex-claim-verification-with-evidence","repo_url":"https://github.com/jifan-chen/fact-checking-via-raw-evidence","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"claim-verification","task_name":"Claim Verification"},{"task_slug":"fact-checking","task_name":"Fact Checking"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.11859","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.11859"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jifan-chen/fact-checking-via-raw-evidence","reach":null}],"summary":{"ran_draft_wrong":4},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"85d54ebef40e394e","entry":"extract_claim_date","repo":"jifan-chen/fact-checking-via-raw-evidence","repo_kind":"official","path":"evidence_retrieval.py","file_url":"https://github.com/jifan-chen/fact-checking-via-raw-evidence/blob/HEAD/evidence_retrieval.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"85d54ebef40e394e"}},{"code_sha256_prefix":"3bc34f31c063e263","entry":"extract_claim_date","repo":"jifan-chen/fact-checking-via-raw-evidence","repo_kind":"official","path":"generate_summarization.py","file_url":"https://github.com/jifan-chen/fact-checking-via-raw-evidence/blob/HEAD/generate_summarization.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3bc34f31c063e263"}},{"code_sha256_prefix":"228097d19f15698f","entry":"filter_corpus_by_timestamp","repo":"jifan-chen/fact-checking-via-raw-evidence","repo_kind":"official","path":"generate_summarization.py","file_url":"https://github.com/jifan-chen/fact-checking-via-raw-evidence/blob/HEAD/generate_summarization.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"228097d19f15698f"}},{"code_sha256_prefix":"a0b165879da7fddf","entry":"map_link2name","repo":"jifan-chen/fact-checking-via-raw-evidence","repo_kind":"official","path":"generate_summarization.py","file_url":"https://github.com/jifan-chen/fact-checking-via-raw-evidence/blob/HEAD/generate_summarization.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a0b165879da7fddf"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}