Browse State-of-the-Art › Fact Checking
Fact Checking
297 papers with code · 7 benchmarks · 12 datasets archive 2025-07-28
Benchmarks archive 2025-07-28
7 leaderboard tables shown for this task, 7 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
Syntology 6 newer papers were judged to report on this task's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 1 placed): SciFact (BEIR) (6 judged, 1 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
12 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
5 subtasks in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 669 · with a code link: 297 · where Syntology ran a sample: 67 (47 with a run with no instrument failure, 20 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (67 of 669 tagged: 47 with a run with no instrument failure, 20 where every run was a failure of Syntology's instrument)
Page 1 of 7: papers 1 to 100 of 669, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
1 May 2017 11 repositories listedIn this paper, we present liar: a new, publicly available dataset for fake news detection.
-
11 Jul 2017 9 repositories listedIdentifying public misinformation is a complicated and challenging task.
-
16 Dec 2021 6 repositories listed Syntology community repositories only · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples) · 1 pointer-only (licence)In this work, we explore the limits of contrastive learning as a way to train unsupervised dense retrievers and show that it leads to strong performance in various retrieval settings.
-
19 May 2021 6 repositories listedThe proliferation of fake news, i. e., news intentionally spread for misinformation, poses a threat to individuals and society.
-
9 May 2024 4 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)To mitigate these issues, we propose OpenFactCheck, a unified framework for building customized automatic fact-checking systems, benchmarking their accuracy, evaluating factuality of LLMs, and verifying claims in a…
-
28 Oct 2019 4 repositories listed Syntology official (archive's flag): 2 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)Currently used metrics for assessing summarization algorithms do not account for whether summaries are factually consistent with source documents.
-
10 Feb 2019 4 repositories listed Syntology 10 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 12 harvested samples) · 1 pointer-only (licence)One of the main reasons is that often the interpretation of the news requires the knowledge of political or social context or 'common sense', which current NLP algorithms are still missing.
-
25 Jul 2023 3 repositories listed Syntology community repositories only · 11 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (of 14 harvested samples)With the above challenges in mind, in this paper, we propose FacTool, a task and domain agnostic framework for detecting factual errors of texts generated by large language models (e. g., ChatGPT).
-
22 May 2023 3 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)Existing datasets for automated fact-checking have substantial limitations, such as relying on artificial claims, lacking annotations for evidence and intermediate reasoning, or including evidence published after the…
-
8 Dec 2021 3 repositories listedLanguage modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
-
2 Dec 2021 3 repositories listed Syntology official (archive's flag): 13 ran · 13 ran (of which 0 constructed an object rather than computing a result; 13 with no instrument failure: 0 honoured, 0 violated, 13 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 13 harvested samples)Our approach outperforms two competitive baselines on three scientific claim verification datasets, with particularly strong performance in zero / few-shot domain adaptation experiments.
-
17 Apr 2021 3 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples)To address this, and to facilitate researchers to broadly evaluate the effectiveness of their models, we introduce Benchmarking-IR (BEIR), a robust and heterogeneous evaluation benchmark for information retrieval.
-
16 Apr 2021 3 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 1 pointer-only (licence)We present KnowledgeEditor, a method which can be used to edit this knowledge and, thus, fix 'bugs' or unexpected predictions without the need for expensive re-training or fine-tuning.
-
31 Dec 2020 3 repositories listedThis paper introduces the task of factual error correction: performing edits to a claim so that the generated rewrite is better supported by evidence.
-
7 Sep 2020 3 repositories listedWhile misinformation and disinformation have been thriving in social media for years, with the emergence of the COVID-19 pandemic, the political and the health misinformation merged, thus elevating the problem to a…
-
4 Sep 2020 3 repositories listed Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)We test both task-specific and general baselines, evaluating downstream performance in addition to the ability of the models to provide provenance.
-
26 Mar 2020 3 repositories listedThe analysis is presented and updated on a publically accessible dashboard (https://usc-melady.github.io/COVID-19-Tweet-Analysis) to track the nature of online discourse and misinformation about COVID-19 on Twitter from…
-
21 Jan 2020 3 repositories listedFinally, the lab offers a fifth task that asks to predict the check-worthiness of the claims made in English political debates and speeches.
-
30 Sep 2019 3 repositories listedThis is a challenging constrained generation task, as the output must be consistent with the new information and fit into the rest of the existing document.
-
8 Mar 2018 3 repositories listedCommunity Question Answering (cQA) forums are very popular nowadays, as they represent effective means for communities around particular topics to share information.
-
3 Feb 2025 2 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)Debunking them requires (1) providing the true context of the image and (2) checking the veracity of the image's caption.
-
6 Aug 2024 2 repositories listedThe increased use of large language models (LLMs) across a variety of real-world applications calls for automatic tools to check the factual accuracy of their outputs, as LLMs often hallucinate.
-
16 Jul 2024 2 repositories listedOur model specifies the expected behavior of each operator with a high-quality gold algorithm, and we develop an optimization framework that reduces cost, while providing accuracy guarantees with respect to a gold…
-
5 Jun 2024 2 repositories listedUnlike previous fallacy detection datasets, Missci (i) focuses on implicit fallacies between the relevant content of the cited publication and the inaccurate claim, and (ii) requires models to verbalize the fallacious…
-
16 Apr 2024 2 repositories listed Syntology official (archive's flag): 2 ran · 2 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 6 unverified (of 8 harvested samples)We release LLM-AggreFact, code for data synthesis, and models.
-
18 Mar 2024 2 repositories listed Syntology official (archive's flag): 4 ran · 4 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples) · 5 pointer-only (licence)Streaming text generation has become a common way of increasing the responsiveness of language model powered applications, such as chat assistants.
-
20 Feb 2024 2 repositories listedSimilar to the FEVER dataset, claims in the "Supports" and "Refutes" categories are also annotated with corresponding evidence sentences sourced from single or multiple pages in Chinese Wikipedia.
-
15 Nov 2023 2 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)The increased use of large language models (LLMs) across a variety of real-world applications calls for mechanisms to verify the factual accuracy of their outputs.
-
4 Jun 2023 2 repositories listedWe run the first systematic evaluation of pre-trained language models for Bulgarian, comparing and contrasting results across the nine tasks in the benchmark.
-
22 May 2023 2 repositories listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples) · 2 pointer-only (licence)Fact-checking real-world claims often requires collecting multiple pieces of evidence and applying complex multi-step reasoning.
-
1 Apr 2023 2 repositories listed
-
2 Mar 2023 2 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
5 Aug 2022 2 repositories listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
15 Oct 2021 2 repositories listed Syntology official (archive's flag): 1 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples)
-
7 Oct 2021 2 repositories listed
-
3 Sep 2021 2 repositories listed
-
5 Jul 2021 2 repositories listed
-
2 Jun 2021 2 repositories listed
-
25 Apr 2021 2 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
18 Apr 2021 2 repositories listed
-
1 Apr 2021 2 repositories listed Syntology community repositories only · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 1 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples)
-
Eating Garlic Prevents COVID-19 Infection: Detecting Misinformation on the Arabic Content of Twitter9 Jan 2021 2 repositories listed
-
19 Oct 2020 2 repositories listed
-
7 Oct 2020 2 repositories listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples)
-
7 Jul 2020 2 repositories listed
-
12 May 2020 2 repositories listed
-
30 Apr 2020 2 repositories listed
-
29 Oct 2019 2 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (of 4 harvested samples)
-
16 Nov 2018 2 repositories listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples)
-
1 Nov 2018 2 repositories listed
-
2 Oct 2018 2 repositories listed
-
17 Sep 2018 2 repositories listed
-
3 Jun 2018 2 repositories listed
-
1 Jul 2017 2 repositories listed
-
16 Nov 2016 2 repositories listed
-
8 Jul 2025 1 repository listed
-
25 Jun 2025 1 repository listed
-
16 Jun 2025 1 repository listed
-
14 Jun 2025 1 repository listed
-
10 Jun 2025 1 repository listed
-
5 Jun 2025 1 repository listed
-
5 Jun 2025 1 repository listed
-
23 May 2025 1 repository listed
-
22 May 2025 1 repository listed
-
21 May 2025 1 repository listed
-
20 May 2025 1 repository listed
-
14 May 2025 1 repository listed
-
13 May 2025 1 repository listed
-
12 May 2025 1 repository listed
-
29 Apr 2025 1 repository listed
-
29 Apr 2025 1 repository listed
-
19 Mar 2025 1 repository listed
-
5 Mar 2025 1 repository listed
-
2 Mar 2025 1 repository listed
-
28 Feb 2025 1 repository listed
-
25 Feb 2025 1 repository listed
-
25 Feb 2025 1 repository listed
-
22 Feb 2025 1 repository listed
-
21 Feb 2025 1 repository listed
-
20 Feb 2025 1 repository listed
-
20 Feb 2025 1 repository listed
-
17 Feb 2025 1 repository listed
-
9 Feb 2025 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 1 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 5 harvested samples) · 5 pointer-only (licence)
-
9 Feb 2025 1 repository listed
-
7 Feb 2025 1 repository listed
-
16 Jan 2025 1 repository listed
-
15 Jan 2025 1 repository listed
-
19 Dec 2024 1 repository listed
-
13 Dec 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples)
-
9 Dec 2024 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 7 unverified (of 10 harvested samples) · 10 pointer-only (licence)
-
6 Dec 2024 1 repository listed
-
29 Nov 2024 1 repository listed
-
18 Nov 2024 1 repository listed
-
28 Oct 2024 1 repository listed
-
28 Oct 2024 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples) · 5 pointer-only (licence)
-
18 Oct 2024 1 repository listed
-
17 Oct 2024 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
16 Oct 2024 1 repository listed
-
15 Oct 2024 1 repository listed
-
14 Oct 2024 1 repository listed
Syntology lines on 28 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections