Question Answering

Description from the archive archive 2025-07-28; Papers-with-Code links inside it are rewritten to this site.

Benchmarks archive 2025-07-28

152 leaderboard tables shown for this task, 142 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted. 10 shown of 152 until expanded.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

413 datasets whose archive record lists this task, ordered by the archive's paper count. 30 shown of 413 until expanded.

Subtasks archive 2025-07-28

19 subtasks in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 4,171 papers with code (10,817 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Attention Is All You Need
  12 Jun 2017 595 repositories listed Syntology ran 600 of 946 samples · 346 unverified · 451 pointer-only (licence)
  The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration.
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  11 Oct 2018 534 repositories listed Syntology ran 204 of 659 samples · 455 unverified · 149 pointer-only (licence)
  We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers.
• Graph Attention Networks
  30 Oct 2017 93 repositories listed Syntology ran 50 of 106 samples · 56 unverified · 43 pointer-only (licence)
  We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior methods based on graph…
• Language Models are Few-Shot Learners
  28 May 2020 67 repositories listed Syntology ran 15 of 65 samples · 50 unverified · 4 pointer-only (licence)
  By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do.
• RoBERTa: A Robustly Optimized BERT Pretraining Approach
  26 Jul 2019 67 repositories listed Syntology ran 22 of 48 samples · 26 unverified · 23 pointer-only (licence)
  Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging.
• LLaMA: Open and Efficient Foundation Language Models
  27 Feb 2023 57 repositories listed Syntology ran 26 of 58 samples · 32 unverified · 4 pointer-only (licence)
  We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters.
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
  23 Oct 2019 57 repositories listed Syntology ran 2 of 31 samples · 29 unverified
  Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP).
• ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
  26 Sep 2019 48 repositories listed Syntology ran 46 of 126 samples · 80 unverified · 22 pointer-only (licence)
  Increasing model size when pretraining natural language representations often results in improved performance on downstream tasks.
• BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
  29 Oct 2019 47 repositories listed Syntology ran 22 of 53 samples · 31 unverified · 7 pointer-only (licence)
  We evaluate a number of noising approaches, finding the best performance by both randomly shuffling the order of the original sentences and using a novel in-filling scheme, where spans of text are replaced with a single…
• Deep contextualized word representations
  15 Feb 2018 46 repositories listed Syntology ran 23 of 58 samples · 35 unverified · 25 pointer-only (licence)
  We introduce a new type of deep contextualized word representation that models both (1) complex characteristics of word use (e.
• End-To-End Memory Networks
  31 Mar 2015 44 repositories listed Syntology ran 2 of 15 samples · 13 unverified · 5 pointer-only (licence)
  For the former our approach is competitive with Memory Networks, but with less supervision.
• DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
  2 Oct 2019 37 repositories listed Syntology ran 19 of 27 samples · 8 unverified
  As Transfer Learning from large-scale pre-trained models becomes more prevalent in Natural Language Processing (NLP), operating these large models in on-the-edge and/or under constrained computational training or…
• Large Batch Optimization for Deep Learning: Training BERT in 76 minutes
  1 Apr 2019 32 repositories listed Syntology ran 1 of 11 samples · 10 unverified · 9 pointer-only (licence)
  In this paper, we first study a principled layerwise adaptation strategy to accelerate training of deep neural networks using large mini-batches.
• XLNet: Generalized Autoregressive Pretraining for Language Understanding
  19 Jun 2019 27 repositories listed Syntology ran 10 of 24 samples · 14 unverified · 3 pointer-only (licence)
  With the capability of modeling bidirectional contexts, denoising autoencoding based pretraining like BERT achieves better performance than pretraining approaches based on autoregressive language modeling.
• Bidirectional Attention Flow for Machine Comprehension
  5 Nov 2016 27 repositories listed Syntology ran 8 of 11 samples · 3 unverified · 7 pointer-only (licence)
  Machine comprehension (MC), answering a query about a given context paragraph, requires modeling complex interactions between the context and the query.
• Distributed Representations of Sentences and Documents
  16 May 2014 27 repositories listed Syntology ran 0 of 4 samples · 4 unverified
  Its construction gives our algorithm the potential to overcome the weaknesses of bag-of-words models.
• Longformer: The Long-Document Transformer
  10 Apr 2020 22 repositories listed Syntology ran 15 of 35 samples · 20 unverified · 5 pointer-only (licence)
  To address this limitation, we introduce the Longformer with an attention mechanism that scales linearly with sequence length, making it easy to process documents of thousands of tokens or longer.
• Language Models are Unsupervised Multitask Learners
  14 Feb 2019 21 repositories listed
  Natural language processing tasks, such as question answering, machine translation, reading comprehension, and summarization, are typically approached with supervised learning on taskspecific datasets.
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
  16 Jun 2016 21 repositories listed Syntology ran 4 of 6 samples · 2 unverified · 6 pointer-only (licence)
  We present the Stanford Question Answering Dataset (SQuAD), a new reading comprehension dataset consisting of 100, 000+ questions posed by crowdworkers on a set of Wikipedia articles, where the answer to each question…
• Pay Attention to MLPs
  17 May 2021 20 repositories listed Syntology ran 34 of 44 samples · 10 unverified · 11 pointer-only (licence)
  Transformers have become one of the most important architectural innovations in deep learning and have enabled many breakthroughs over the past few years.
• A simple neural network module for relational reasoning
  5 Jun 2017 20 repositories listed Syntology ran 3 of 7 samples · 4 unverified · 1 pointer-only (licence)
  Relational reasoning is a central component of generally intelligent behavior, but has proven difficult for neural networks to learn.
• Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
  19 Feb 2015 20 repositories listed Syntology ran 1 of 3 samples · 2 unverified · 2 pointer-only (licence)
  One long-term goal of machine learning research is to produce methods that are applicable to reasoning and natural language, in particular building an intelligent dialogue agent.
• Llama 2: Open Foundation and Fine-Tuned Chat Models
  18 Jul 2023 19 repositories listed Syntology ran 31 of 52 samples · 21 unverified · 16 pointer-only (licence)
  In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters.
• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  28 Jan 2022 19 repositories listed Syntology ran 2 of 7 samples · 5 unverified
  We explore how generating a chain of thought -- a series of intermediate reasoning steps -- significantly improves the ability of large language models to perform complex reasoning.
• Dense Passage Retrieval for Open-Domain Question Answering
  10 Apr 2020 19 repositories listed Syntology ran 10 of 14 samples · 4 unverified · 9 pointer-only (licence)
  Open-domain question answering relies on efficient passage retrieval to select candidate contexts, where traditional sparse vector space models, such as TF-IDF or BM25, are the de facto method.
• ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
  23 Mar 2020 19 repositories listed Syntology ran 26 of 40 samples · 14 unverified · 10 pointer-only (licence)
  Then, instead of training a model that predicts the original identities of the corrupted tokens, we train a discriminative model that predicts whether each token in the corrupted input was replaced by a generator sample…
• ERNIE: Enhanced Representation through Knowledge Integration
  19 Apr 2019 19 repositories listed Syntology ran 0 of 7 samples · 7 unverified
  We present a novel language representation model enhanced by knowledge called ERNIE (Enhanced Representation through kNowledge IntEgration).
• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
  25 Jan 2019 19 repositories listed Syntology ran 4 of 25 samples · 21 unverified · 1 pointer-only (licence)
  Biomedical text mining is becoming increasingly important as the number of biomedical documents rapidly grows.
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  22 May 2020 18 repositories listed Syntology ran 4 of 6 samples · 2 unverified
  Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks.
• QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
  23 Apr 2018 15 repositories listed Syntology ran 7 of 19 samples · 12 unverified · 2 pointer-only (licence)
  On the SQuAD dataset, our model is 3x to 13x faster in training and 4x to 9x faster in inference, while achieving equivalent accuracy to recurrent models.

Syntology lines on 29 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.