Common Sense Reasoning

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

24 leaderboard tables shown for this task, 24 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted. 10 shown of 24 until expanded.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

56 datasets whose archive record lists this task, ordered by the archive's paper count. 30 shown of 56 until expanded.

Subtasks archive 2025-07-28

17 subtasks in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 325 papers with code (939 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  11 Oct 2018 534 repositories listed Syntology ran 204 of 659 samples · 455 unverified · 149 pointer-only (licence)
  We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers.
• Language Models are Few-Shot Learners
  28 May 2020 67 repositories listed Syntology ran 15 of 65 samples · 50 unverified · 4 pointer-only (licence)
  By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do.
• RoBERTa: A Robustly Optimized BERT Pretraining Approach
  26 Jul 2019 67 repositories listed Syntology ran 22 of 48 samples · 26 unverified · 23 pointer-only (licence)
  Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging.
• LLaMA: Open and Efficient Foundation Language Models
  27 Feb 2023 57 repositories listed Syntology ran 26 of 58 samples · 32 unverified · 4 pointer-only (licence)
  We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters.
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
  23 Oct 2019 57 repositories listed Syntology ran 2 of 31 samples · 29 unverified
  Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP).
• ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
  26 Sep 2019 48 repositories listed Syntology ran 46 of 126 samples · 80 unverified · 22 pointer-only (licence)
  Increasing model size when pretraining natural language representations often results in improved performance on downstream tasks.
• Mamba: Linear-Time Sequence Modeling with Selective State Spaces
  1 Dec 2023 35 repositories listed Syntology ran 18 of 62 samples · 44 unverified · 28 pointer-only (licence)
  Foundation models, now powering most of the exciting applications in deep learning, are almost universally based on the Transformer architecture and its core attention module.
• Language Models are Unsupervised Multitask Learners
  14 Feb 2019 21 repositories listed
  Natural language processing tasks, such as question answering, machine translation, reading comprehension, and summarization, are typically approached with supervised learning on taskspecific datasets.
• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  28 Jan 2022 19 repositories listed Syntology ran 2 of 7 samples · 5 unverified
  We explore how generating a chain of thought -- a series of intermediate reasoning steps -- significantly improves the ability of large language models to perform complex reasoning.
• A Neural Conversational Model
  19 Jun 2015 18 repositories listed Syntology ran 2 of 4 samples · 2 unverified · 3 pointer-only (licence)
  We find that this straightforward model can generate simple conversations given a large conversational training dataset.
• DeBERTa: Decoding-enhanced BERT with Disentangled Attention
  5 Jun 2020 14 repositories listed Syntology ran 4 of 13 samples · 9 unverified · 3 pointer-only (licence)
  Recent progress in pre-trained neural language models has significantly improved the performance of many natural language processing (NLP) tasks.
• AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
  1 Jun 2023 12 repositories listed Syntology ran 12 of 18 samples · 6 unverified · 2 pointer-only (licence)
  We propose Activation-aware Weight Quantization (AWQ), a hardware-friendly approach for LLM low-bit weight-only quantization.
• GPT-4 Technical Report
  15 Mar 2023 11 repositories listed Syntology ran 2 of 5 samples · 3 unverified · 1 pointer-only (licence)
  We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs.
• WinoGrande: An Adversarial Winograd Schema Challenge at Scale
  24 Jul 2019 10 repositories listed
  The key steps of the dataset construction consist of (1) a carefully designed crowdsourcing procedure, followed by (2) systematic bias reduction using a novel AfLite algorithm that generalizes human-detectable word…
• LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention
  2 Oct 2020 9 repositories listed Syntology ran 3 of 10 samples · 7 unverified
  In this paper, we propose new pretrained contextualized representations of words and entities based on the bidirectional transformer.
• Finetuned Language Models Are Zero-Shot Learners
  3 Sep 2021 8 repositories listed Syntology ran 0 of 1 samples · 1 unverified
  We show that instruction tuning -- finetuning language models on a collection of tasks described via instructions -- substantially improves zero-shot performance on unseen tasks.
• mT5: A massively multilingual pre-trained text-to-text transformer
  22 Oct 2020 8 repositories listed Syntology ran 0 of 13 samples · 13 unverified
  The recent "Text-to-Text Transfer Transformer" (T5) leveraged a unified text-to-text format and scale to attain state-of-the-art results on a wide variety of English-language NLP tasks.
• PaLM: Scaling Language Modeling with Pathways
  5 Apr 2022 7 repositories listed Syntology ran 30 of 37 samples · 7 unverified
  To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.
• Qwen2.5 Technical Report
  19 Dec 2024 6 repositories listed Syntology ran 1 of 3 samples · 2 unverified
  In addition, for hosted solutions, the proprietary models currently include two mixture-of-experts (MoE) variants: Qwen2.
• Mixtral of Experts
  8 Jan 2024 6 repositories listed Syntology ran 5 of 5 samples · 0 unverified
  In particular, Mixtral vastly outperforms Llama 2 70B on mathematics, code generation, and multilingual benchmarks.
• Mistral 7B
  10 Oct 2023 6 repositories listed Syntology ran 9 of 11 samples · 2 unverified · 1 pointer-only (licence)
  We introduce Mistral 7B v0.
• SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot
  2 Jan 2023 6 repositories listed Syntology ran 2 of 12 samples · 10 unverified · 9 pointer-only (licence)
  We show for the first time that large-scale generative pretrained transformer (GPT) family models can be pruned to at least 50% sparsity in one-shot, without any retraining, at minimal loss of accuracy.
• Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
  9 Jun 2022 6 repositories listed Syntology ran 0 of 3 samples · 3 unverified
  BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
• QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering
  13 Apr 2021 6 repositories listed Syntology ran 1 of 25 samples · 24 unverified · 4 pointer-only (licence)
  The problem of answering questions using knowledge from pre-trained language models (LMs) and knowledge graphs (KGs) presents two challenges: given a QA context (question and answer choice), methods need to (i) identify…
• Temporal Relational Reasoning in Videos
  22 Nov 2017 5 repositories listed Syntology ran 2 of 3 samples · 1 unverified · 3 pointer-only (licence)
  Temporal relational reasoning, the ability to link meaningful transformations of objects or entities over time, is a fundamental property of intelligent species.
• The "something something" video database for learning and evaluating visual common sense
  13 Jun 2017 5 repositories listed
  Neural networks trained on datasets such as ImageNet have led to major advances in visual object classification.
• Gated Delta Networks: Improving Mamba2 with Delta Rule
  9 Dec 2024 4 repositories listed Syntology ran 5 of 13 samples · 8 unverified · 7 pointer-only (licence)
  Linear Transformers have gained attention as efficient alternatives to standard Transformers, but their performance in retrieval and long-context tasks has been limited.
• Smart Agent-Based Modeling: On the Use of Large Language Models in Computer Simulations
  10 Nov 2023 4 repositories listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  ABM's strength lies in its bottom-up methodology, illuminating emergent phenomena by modeling the behaviors of individual components of a system.
• Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
  3 Apr 2023 4 repositories listed
  How do large language models (LLMs) develop and evolve over the course of training?
• Large Language Models are Zero-Shot Reasoners
  24 May 2022 4 repositories listed Syntology ran 0 of 4 samples · 4 unverified · 1 pointer-only (licence)
  Pretrained large language models (LLMs) are widely used in many sub-fields of natural language processing (NLP) and generally known as excellent few-shot learners with task-specific exemplars.

Syntology lines on 26 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.