Logical Fallacies

Benchmarks archive 2025-07-28

1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

2 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

14 shown of 14 papers with code (34 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Scaling Language Models: Methods, Analysis & Insights from Training Gopher
  8 Dec 2021 3 repositories listed
  Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
• Logical Fallacy Detection
  28 Feb 2022 2 repositories listed
  In this paper, we propose the task of logical fallacy detection, and provide a new dataset (Logic) of logical fallacies generally found in text, together with an additional challenge set for detecting logical fallacies…
• Leveraging Context for Multimodal Fallacy Classification in Political Debates
  21 Jul 2025 1 repository listed
  In this paper, we present our submission to the MM-ArgFallacy2025 shared task, which aims to advance research in multimodal argument mining, focusing on logical fallacies in political debates.
• Socrates or Smartypants: Testing Logic Reasoning Capabilities of Large Language Models with Logic Programming-based Test Oracles
  9 Apr 2025 1 repository listed
  Large Language Models (LLMs) have achieved significant progress in language understanding and reasoning.
• Large Language Models Are Better Logical Fallacy Reasoners with Counterargument, Explanation, and Goal-Aware Prompt Formulation
  30 Mar 2025 1 repository listed
  The advancement of Large Language Models (LLMs) has greatly improved our ability to process complex language.
• RuozhiBench: Evaluating LLMs with Logical Fallacies and Misleading Premises
  18 Feb 2025 1 repository listed
  Recent advances in large language models (LLMs) have shown that they can answer questions requiring complex reasoning.
• Boosting Logical Fallacy Reasoning in LLMs via Logical Structure Tree
  15 Oct 2024 1 repository listed
  We further develop two strategies to incorporate the logical structure tree into LLMs for fallacy reasoning.
• Grounding Fallacies Misrepresenting Scientific Publications in Evidence
  23 Aug 2024 1 repository listed
  Health-related misinformation claims often falsely cite a credible biomedical publication as evidence, which superficially appears to support the false claim.
• A Logical Fallacy-Informed Framework for Argument Generation
  7 Aug 2024 1 repository listed
  Despite the remarkable performance of Large Language Models (LLMs) in natural language processing tasks, they still struggle with generating logically sound arguments, resulting in potential risks such as spreading…
• OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems
  21 Feb 2024 1 repository listed Syntology ran 5 of 6 samples · 1 unverified
  Notably, the best-performing model, GPT-4V, attains an average score of 17.
• A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoning
  14 Nov 2023 1 repository listed
  In this paper, we take a closer look at the self-verification abilities of LLMs in the context of logical reasoning, focusing on their ability to identify logical fallacies accurately.
• How susceptible are LLMs to Logical Fallacies?
  18 Aug 2023 1 repository listed
  Then, it evaluates the debater's performance in logical reasoning by contrasting the scenario where the persuader employs logical fallacies against one where logical reasoning is used.
• Case-Based Reasoning with Language Models for Classification of Logical Fallacies
  27 Jan 2023 1 repository listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  The ease and speed of spreading misinformation and propaganda on the Web motivate the need to develop trustworthy technology for detecting fallacies in natural language arguments.
• Robust and Explainable Identification of Logical Fallacies in Natural Language Arguments
  12 Dec 2022 1 repository listed
  Our three-stage framework natively consolidates prior datasets and methods from existing tasks, like propaganda detection, serving as an overarching evaluation testbed.

Syntology lines on 2 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.