Browse State-of-the-Art › Question Answering
Question Answering
4,171 papers with code · 142 benchmarks · 413 datasets archive 2025-07-28
Question answering can be segmented into domain-specific tasks like community question answering and knowledge-base question answering. Popular benchmark datasets for evaluation question answering systems include SQuAD, HotPotQA, bAbI, TriviaQA, WikiQA, and many others. Models for question answering are typically evaluated on metrics like EM and F1. Some recent top performing models are T5 and XLNet.
( Image credit: SQuAD )
Description from the archive archive 2025-07-28; Papers-with-Code links inside it are rewritten to this site.
Benchmarks archive 2025-07-28
152 leaderboard tables shown for this task, 142 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted. 10 shown of 152 until expanded.
Syntology 181 newer papers were judged to report on this task's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 16 placed): SQuAD1.1 (3 judged, 0 placed), HotpotQA (19 judged, 0 placed), PIQA (59 judged, 9 placed), BoolQ (31 judged, 0 placed), COPA (7 judged, 0 placed), TriviaQA (20 judged, 0 placed), SQuAD1.1 dev (2 judged, 0 placed), OpenBookQA (22 judged, 1 placed), WebQuestions (1 judged, 0 placed), TruthfulQA (22 judged, 1 placed), PubMedQA (8 judged, 0 placed), MedQA (17 judged, 3 placed), SIQA (7 judged, 0 placed), StoryCloze (1 judged, 0 placed), Quora Question Pairs (1 judged, 0 placed), CNN / Daily Mail (1 judged, 0 placed), StrategyQA (3 judged, 0 placed), NarrativeQA (2 judged, 0 placed), Bamboogle (11 judged, 1 placed), OBQA (25 judged, 0 placed), and 11 more leaderboards in the JSON twin. Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
413 datasets whose archive record lists this task, ordered by the archive's paper count. 30 shown of 413 until expanded.
Subtasks archive 2025-07-28
19 subtasks in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 10,817 · with a code link: 4,171 · where Syntology ran a sample: 1,274 (1,037 with a run with no instrument failure, 237 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (1,274 of 10,817 tagged: 1,037 with a run with no instrument failure, 237 where every run was a failure of Syntology's instrument)
Page 1 of 109: papers 1 to 100 of 10,817, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
12 Jun 2017 595 repositories listed Syntology official: no sample here; runs from other or unrecorded repositories · 610 ran (of which 293 constructed an object rather than computing a result; 529 with no instrument failure: 45 honoured, 15 violated, 469 with no contract checked; 81 where Syntology's instrument failed) · 336 unverified (of 946 harvested samples) · 451 pointer-only (licence)The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration.
-
11 Oct 2018 534 repositories listed Syntology official: no sample here; runs from other or unrecorded repositories · 300 ran (of which 75 constructed an object rather than computing a result; 234 with no instrument failure: 18 honoured, 5 violated, 211 with no contract checked; 66 where Syntology's instrument failed) · 359 unverified (of 659 harvested samples) · 164 pointer-only (licence)We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers.
-
30 Oct 2017 93 repositories listed Syntology community repositories only · 61 ran (of which 28 constructed an object rather than computing a result; 52 with no instrument failure: 1 honoured, 2 violated, 49 with no contract checked; 9 where Syntology's instrument failed) · 45 unverified (of 106 harvested samples) · 46 pointer-only (licence)We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior methods based on graph…
-
28 May 2020 67 repositories listed Syntology community repositories only · 45 ran (of which 0 constructed an object rather than computing a result; 37 with no instrument failure: 3 honoured, 2 violated, 32 with no contract checked; 8 where Syntology's instrument failed) · 20 unverified (of 65 harvested samples) · 7 pointer-only (licence)By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do.
-
26 Jul 2019 67 repositories listed Syntology community repositories only · 37 ran (of which 11 constructed an object rather than computing a result; 36 with no instrument failure: 2 honoured, 0 violated, 34 with no contract checked; 1 where Syntology's instrument failed) · 11 unverified (of 48 harvested samples) · 24 pointer-only (licence)Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging.
-
27 Feb 2023 57 repositories listed Syntology official: no sample here; runs from other or unrecorded repositories · 37 ran (of which 9 constructed an object rather than computing a result; 25 with no instrument failure: 3 honoured, 0 violated, 22 with no contract checked; 12 where Syntology's instrument failed) · 21 unverified (of 58 harvested samples) · 4 pointer-only (licence)We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters.
-
23 Oct 2019 57 repositories listed Syntology 21 ran (of which 0 constructed an object rather than computing a result; 20 with no instrument failure: 1 honoured, 0 violated, 19 with no contract checked; 1 where Syntology's instrument failed) · 10 unverified (of 31 harvested samples)Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP).
-
26 Sep 2019 48 repositories listed Syntology official (archive's flag): 6 ran · 81 ran (of which 17 constructed an object rather than computing a result; 56 with no instrument failure: 4 honoured, 0 violated, 52 with no contract checked; 25 where Syntology's instrument failed) · 45 unverified (of 126 harvested samples) · 28 pointer-only (licence)Increasing model size when pretraining natural language representations often results in improved performance on downstream tasks.
-
29 Oct 2019 47 repositories listed Syntology 40 ran (of which 7 constructed an object rather than computing a result; 30 with no instrument failure: 1 honoured, 2 violated, 27 with no contract checked; 10 where Syntology's instrument failed) · 13 unverified (of 53 harvested samples) · 13 pointer-only (licence)We evaluate a number of noising approaches, finding the best performance by both randomly shuffling the order of the original sentences and using a novel in-filling scheme, where spans of text are replaced with a single…
-
15 Feb 2018 46 repositories listed Syntology 30 ran (of which 12 constructed an object rather than computing a result; 24 with no instrument failure: 2 honoured, 0 violated, 22 with no contract checked; 6 where Syntology's instrument failed) · 28 unverified (of 58 harvested samples) · 25 pointer-only (licence)We introduce a new type of deep contextualized word representation that models both (1) complex characteristics of word use (e. g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i. e.…
-
31 Mar 2015 44 repositories listed Syntology community repositories only · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 11 unverified (of 15 harvested samples) · 5 pointer-only (licence)For the former our approach is competitive with Memory Networks, but with less supervision.
-
2 Oct 2019 37 repositories listed Syntology official (archive's flag): 1 ran · 21 ran (of which 5 constructed an object rather than computing a result; 12 with no instrument failure: 3 honoured, 1 violated, 8 with no contract checked; 9 where Syntology's instrument failed) · 6 unverified (of 27 harvested samples) · 2 pointer-only (licence)As Transfer Learning from large-scale pre-trained models becomes more prevalent in Natural Language Processing (NLP), operating these large models in on-the-edge and/or under constrained computational training or…
-
1 Apr 2019 32 repositories listed Syntology community repositories only · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 5 unverified (of 11 harvested samples) · 9 pointer-only (licence)In this paper, we first study a principled layerwise adaptation strategy to accelerate training of deep neural networks using large mini-batches.
-
19 Jun 2019 27 repositories listed Syntology official (archive's flag): 1 ran · 15 ran (of which 2 constructed an object rather than computing a result; 10 with no instrument failure: 2 honoured, 0 violated, 8 with no contract checked; 5 where Syntology's instrument failed) · 9 unverified (of 24 harvested samples) · 4 pointer-only (licence)With the capability of modeling bidirectional contexts, denoising autoencoding based pretraining like BERT achieves better performance than pretraining approaches based on autoregressive language modeling.
-
5 Nov 2016 27 repositories listed Syntology official (archive's flag): 1 ran · 9 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 2 violated, 1 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified (of 11 harvested samples) · 7 pointer-only (licence)Machine comprehension (MC), answering a query about a given context paragraph, requires modeling complex interactions between the context and the query.
-
16 May 2014 27 repositories listed Syntology 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples)Its construction gives our algorithm the potential to overcome the weaknesses of bag-of-words models.
-
10 Apr 2020 22 repositories listed Syntology official (archive's flag): 9 ran · 22 ran (of which 4 constructed an object rather than computing a result; 14 with no instrument failure: 1 honoured, 1 violated, 12 with no contract checked; 8 where Syntology's instrument failed) · 13 unverified (of 35 harvested samples) · 5 pointer-only (licence)To address this limitation, we introduce the Longformer with an attention mechanism that scales linearly with sequence length, making it easy to process documents of thousands of tokens or longer.
-
14 Feb 2019 21 repositories listedNatural language processing tasks, such as question answering, machine translation, reading comprehension, and summarization, are typically approached with supervised learning on taskspecific datasets.
-
16 Jun 2016 21 repositories listed Syntology community repositories only · 4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples) · 6 pointer-only (licence)We present the Stanford Question Answering Dataset (SQuAD), a new reading comprehension dataset consisting of 100, 000+ questions posed by crowdworkers on a set of Wikipedia articles, where the answer to each question…
-
17 May 2021 20 repositories listed Syntology 34 ran (of which 15 constructed an object rather than computing a result; 31 with no instrument failure: 1 honoured, 4 violated, 26 with no contract checked; 3 where Syntology's instrument failed) · 10 unverified (of 44 harvested samples) · 11 pointer-only (licence)Transformers have become one of the most important architectural innovations in deep learning and have enabled many breakthroughs over the past few years.
-
5 Jun 2017 20 repositories listed Syntology 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified (of 7 harvested samples) · 1 pointer-only (licence)Relational reasoning is a central component of generally intelligent behavior, but has proven difficult for neural networks to learn.
-
19 Feb 2015 20 repositories listed Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples) · 2 pointer-only (licence)One long-term goal of machine learning research is to produce methods that are applicable to reasoning and natural language, in particular building an intelligent dialogue agent.
-
18 Jul 2023 19 repositories listed Syntology community repositories only · 33 ran (of which 7 constructed an object rather than computing a result; 20 with no instrument failure: 1 honoured, 1 violated, 18 with no contract checked; 13 where Syntology's instrument failed) · 19 unverified (of 52 harvested samples) · 20 pointer-only (licence)In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters.
-
28 Jan 2022 19 repositories listed Syntology 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples) · 1 pointer-only (licence)We explore how generating a chain of thought -- a series of intermediate reasoning steps -- significantly improves the ability of large language models to perform complex reasoning.
-
10 Apr 2020 19 repositories listed Syntology official: harvested, nothing ran · 11 ran (of which 6 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (of 14 harvested samples) · 9 pointer-only (licence)Open-domain question answering relies on efficient passage retrieval to select candidate contexts, where traditional sparse vector space models, such as TF-IDF or BM25, are the de facto method.
-
23 Mar 2020 19 repositories listed Syntology official: no sample here; runs from other or unrecorded repositories · 31 ran (of which 7 constructed an object rather than computing a result; 18 with no instrument failure: 2 honoured, 2 violated, 14 with no contract checked; 13 where Syntology's instrument failed) · 9 unverified (of 40 harvested samples) · 10 pointer-only (licence)Then, instead of training a model that predicts the original identities of the corrupted tokens, we train a discriminative model that predicts whether each token in the corrupted input was replaced by a generator sample…
-
19 Apr 2019 19 repositories listed Syntology community repositories only · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples)We present a novel language representation model enhanced by knowledge called ERNIE (Enhanced Representation through kNowledge IntEgration).
-
25 Jan 2019 19 repositories listed Syntology official: no sample here; runs from other or unrecorded repositories · 15 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 1 honoured, 0 violated, 10 with no contract checked; 4 where Syntology's instrument failed) · 10 unverified (of 25 harvested samples) · 3 pointer-only (licence)Biomedical text mining is becoming increasingly important as the number of biomedical documents rapidly grows.
-
22 May 2020 18 repositories listed Syntology 6 ran (of which 3 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples)Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks.
-
23 Apr 2018 15 repositories listed Syntology 14 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 3 honoured, 0 violated, 7 with no contract checked; 4 where Syntology's instrument failed) · 5 unverified (of 19 harvested samples) · 3 pointer-only (licence)On the SQuAD dataset, our model is 3x to 13x faster in training and 4x to 9x faster in inference, while achieving equivalent accuracy to recurrent models.
-
28 Jul 2020 14 repositories listed Syntology official (archive's flag): 1 ran · 10 ran (of which 1 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 8 where Syntology's instrument failed) · 5 unverified (of 15 harvested samples) · 11 pointer-only (licence)
-
5 Jun 2020 14 repositories listed Syntology official: no sample here; runs from other or unrecorded repositories · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 9 unverified (of 13 harvested samples) · 3 pointer-only (licence)
-
2 Apr 2019 13 repositories listed Syntology community repositories only · 13 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 1 violated, 10 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 15 harvested samples) · 15 pointer-only (licence)
-
11 Jun 2018 13 repositories listed
-
15 Mar 2023 11 repositories listed Syntology community repositories only · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 2 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 1 pointer-only (licence)
-
4 Mar 2022 11 repositories listed Syntology community repositories only · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples)
-
6 Aug 2019 11 repositories listed Syntology 10 ran (of which 6 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 2 where Syntology's instrument failed) · 24 unverified (of 34 harvested samples) · 34 pointer-only (licence)
-
11 Jun 2018 11 repositories listed
-
8 Sep 2017 11 repositories listed Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
7 Feb 2021 10 repositories listed Syntology official: no sample here; runs from other or unrecorded repositories · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples) · 1 pointer-only (licence)
-
13 Jan 2020 10 repositories listed Syntology community repositories only · 6 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified (of 8 harvested samples)
-
16 Nov 2019 10 repositories listed Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
23 Sep 2019 10 repositories listed Syntology official: harvested, nothing ran · 0 ran · 4 unverified (of 4 harvested samples) · 4 pointer-only (licence)
-
17 Sep 2019 10 repositories listed Syntology community repositories only · 12 ran (of which 2 constructed an object rather than computing a result; 7 with no instrument failure: 4 honoured, 0 violated, 3 with no contract checked; 5 where Syntology's instrument failed) · 35 unverified (of 47 harvested samples) · 15 pointer-only (licence)
-
24 Jul 2019 10 repositories listed
-
16 Jul 2019 10 repositories listed Syntology 8 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 2 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 4 unverified (of 12 harvested samples) · 9 pointer-only (licence)
-
31 Mar 2017 10 repositories listed Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
4 Mar 2016 10 repositories listed Syntology 7 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 7 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples) · 7 pointer-only (licence)
-
20 Oct 2022 9 repositories listed Syntology 8 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 7 where Syntology's instrument failed) · 9 unverified (of 17 harvested samples) · 2 pointer-only (licence)
-
6 Oct 2022 9 repositories listed Syntology official (archive's flag): 3 ran · 15 ran (of which 9 constructed an object rather than computing a result; 10 with no instrument failure: 1 honoured, 0 violated, 9 with no contract checked; 5 where Syntology's instrument failed) · 19 unverified (of 34 harvested samples) · 6 pointer-only (licence)
-
2 Oct 2020 9 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 2 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 7 unverified (of 10 harvested samples)
-
20 Aug 2019 9 repositories listed Syntology community repositories only · 4 ran (of which 4 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 11 unverified; every one of the 4 samples that ran constructed an object rather than computing a result (of 15 harvested samples) · 3 pointer-only (licence)
-
8 May 2019 9 repositories listed
-
3 Sep 2021 8 repositories listed Syntology official: harvested for another paper · 0 ran · 1 unverified (of 1 harvested sample)
-
11 Jan 2021 8 repositories listed Syntology official: harvested, nothing ran · 7 ran (of which 2 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 4 where Syntology's instrument failed) · 6 unverified (of 13 harvested samples) · 8 pointer-only (licence)
-
22 Oct 2020 8 repositories listed Syntology official (archive's flag): 5 ran · 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 13 harvested samples)
-
2 Jul 2020 8 repositories listed
-
5 Apr 2020 8 repositories listed Syntology community repositories only · 14 ran (of which 0 constructed an object rather than computing a result; 14 with no instrument failure: 0 honoured, 0 violated, 14 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 14 harvested samples)
-
22 Mar 2019 8 repositories listed Syntology official (archive's flag): 8 ran · 19 ran (of which 0 constructed an object rather than computing a result; 14 with no instrument failure: 3 honoured, 0 violated, 11 with no contract checked; 5 where Syntology's instrument failed) · 8 unverified (of 27 harvested samples) · 9 pointer-only (licence)
-
21 Feb 2023 7 repositories listed Syntology official (archive's flag): 2 ran · 5 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 4 pointer-only (licence)
-
5 Dec 2020 7 repositories listed
-
6 Apr 2020 7 repositories listed
-
10 Nov 2019 7 repositories listed Syntology 6 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples) · 4 pointer-only (licence)
-
25 Oct 2019 7 repositories listed Syntology community repositories only · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples)
-
25 Sep 2019 7 repositories listed Syntology community repositories only · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 2 pointer-only (licence)
-
25 Jun 2019 7 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
23 Apr 2019 7 repositories listed Syntology community repositories only · 5 ran (of which 4 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples)
-
8 Jan 2024 6 repositories listed Syntology 5 ran (of which 5 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 5 samples that ran constructed an object rather than computing a result (of 5 harvested samples)
-
16 Nov 2023 6 repositories listed Syntology community repositories only · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples) · 2 pointer-only (licence)
-
17 Oct 2023 6 repositories listed Syntology official (archive's flag): 8 ran · 11 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 1 violated, 2 with no contract checked; 8 where Syntology's instrument failed) · 3 unverified (of 14 harvested samples) · 6 pointer-only (licence)
-
10 Oct 2023 6 repositories listed Syntology official (archive's flag): 9 ran · 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 11 harvested samples) · 2 pointer-only (licence)
-
6 Jul 2023 6 repositories listed Syntology official: harvested, nothing ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 4 harvested samples) · 2 pointer-only (licence)
-
17 May 2023 6 repositories listed Syntology official (archive's flag): 2 ran · 19 ran (of which 6 constructed an object rather than computing a result; 17 with no instrument failure: 0 honoured, 0 violated, 17 with no contract checked; 2 where Syntology's instrument failed) · 5 unverified (of 24 harvested samples) · 1 pointer-only (licence)
-
2 Jan 2023 6 repositories listed Syntology official (archive's flag): 2 ran · 9 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (of 12 harvested samples) · 9 pointer-only (licence)
-
16 Dec 2021 6 repositories listed Syntology community repositories only · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples) · 1 pointer-only (licence)
-
13 Apr 2021 6 repositories listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 1 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 17 unverified (of 25 harvested samples) · 5 pointer-only (licence)
-
22 Feb 2021 6 repositories listed
-
19 Jun 2020 6 repositories listed Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
10 Feb 2020 6 repositories listed Syntology community repositories only · 4 ran (of which 1 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 2 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples)
-
24 Jul 2019 6 repositories listed Syntology official (archive's flag): 2 ran · 9 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 4 where Syntology's instrument failed) · 6 unverified (of 15 harvested samples) · 6 pointer-only (licence)
-
10 Dec 2018 6 repositories listed
-
20 Jun 2018 6 repositories listed Syntology official (archive's flag): 9 ran · 13 ran (of which 0 constructed an object rather than computing a result; 12 with no instrument failure: 0 honoured, 0 violated, 12 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 13 harvested samples) · 4 pointer-only (licence)
-
10 Dec 2017 6 repositories listed
-
21 Nov 2017 6 repositories listed
-
4 Aug 2017 6 repositories listed
-
5 Nov 2016 6 repositories listed
-
19 Nov 2015 6 repositories listed Syntology 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 4 harvested samples) · 1 pointer-only (licence)
-
2 Oct 2023 5 repositories listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 2 where Syntology's instrument failed) · 4 unverified (of 10 harvested samples)
-
1 Sep 2023 5 repositories listed Syntology official (archive's flag): 10 ran · 15 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 3 honoured, 1 violated, 5 with no contract checked; 6 where Syntology's instrument failed) · 5 unverified (of 20 harvested samples) · 15 pointer-only (licence)
-
24 Jul 2023 5 repositories listed Syntology 6 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 4 unverified (of 10 harvested samples) · 10 pointer-only (licence)
-
2 Mar 2023 5 repositories listed
-
7 Oct 2022 5 repositories listed Syntology official: no sample here; runs from other or unrecorded repositories · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples) · 2 pointer-only (licence)
-
23 May 2022 5 repositories listed
-
29 Apr 2022 5 repositories listed Syntology 18 ran (of which 6 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 1 violated, 10 with no contract checked; 7 where Syntology's instrument failed) · 6 unverified (of 24 harvested samples) · 8 pointer-only (licence)
-
13 Jun 2021 5 repositories listed Syntology official (archive's flag): 5 ran · 11 ran (of which 3 constructed an object rather than computing a result; 7 with no instrument failure: 2 honoured, 1 violated, 4 with no contract checked; 4 where Syntology's instrument failed) · 7 unverified (of 18 harvested samples) · 5 pointer-only (licence)
Syntology lines on 84 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections