Browse State-of-the-Art › Benchmarking
Benchmarking
2,658 papers with code · 2 benchmarks · 10 datasets archive 2025-07-28
Benchmarks archive 2025-07-28
2 leaderboard tables shown for this task, 2 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| CloudEval-YAML (1 row) | GPT-4 Turbo | CloudEval-YAML: A Practical Benchmark for Cloud Configuration Generation | code | — | Compare |
| Wiki-40B (1 row) | OutEffHop-Bert_base | Outlier-Efficient Hopfield Layers for Large Transformer-Based Models | code | Syntology official (archive's flag): 10 ran · 10 ran (of which 5 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 5 where Syntology's instrument failed) · 3 unverified | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
10 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 5,548 · with a code link: 2,658 · where Syntology ran a sample: 749 (603 with a run with no instrument failure, 146 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (749 of 5,548 tagged: 603 with a run with no instrument failure, 146 where every run was a failure of Syntology's instrument)
Page 1 of 56: papers 1 to 100 of 5,548, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
17 Jun 2019 142 repositories listed Syntology community repositories only · 62 ran (of which 0 constructed an object rather than computing a result; 53 with no instrument failure: 1 honoured, 2 violated, 50 with no contract checked; 9 where Syntology's instrument failed) · 20 unverified (of 82 harvested samples) · 12 pointer-only (licence)In this paper, we introduce the various features of this toolbox.
-
26 Feb 2021 82 repositories listed Syntology official: no sample here; runs from other or unrecorded repositories · 16 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 14 where Syntology's instrument failed) · 4 unverified (of 20 harvested samples) · 16 pointer-only (licence)State-of-the-art computer vision systems are trained to predict a fixed set of predetermined object categories.
-
25 Aug 2017 37 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)We present Fashion-MNIST, a new dataset comprising of 28x28 grayscale images of 70, 000 fashion products from 10 categories, with 7, 000 images per category.
-
20 Nov 2014 24 repositories listed Syntology 13 ran (of which 7 constructed an object rather than computing a result; 9 with no instrument failure: 1 honoured, 0 violated, 8 with no contract checked; 4 where Syntology's instrument failed) · 19 unverified (of 32 harvested samples) · 27 pointer-only (licence)We propose a novel paradigm for evaluating image descriptions that uses human consensus.
-
11 Feb 2019 23 repositories listed Syntology community repositories only · 10 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 6 where Syntology's instrument failed) · 5 unverified (of 15 harvested samples) · 14 pointer-only (licence)In this paper, we propose the StarCraft Multi-Agent Challenge (SMAC) as a benchmark problem to fill this gap.
-
2 Mar 2020 15 repositories listed Syntology official (archive's flag): 3 ran · 10 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 1 where Syntology's instrument failed) · 13 unverified (of 23 harvested samples) · 1 pointer-only (licence)In the last few years, graph neural networks (GNNs) have become the standard toolkit for analyzing and learning from data on graphs.
-
22 Apr 2016 15 repositories listed Syntology community repositories only · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples)Recently, researchers have made significant progress combining the advances in deep learning for learning feature representations with reinforcement learning.
-
28 Mar 2019 14 repositories listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples)Then we propose a new dataset called ImageNet-P which enables researchers to benchmark a classifier's robustness to common perturbations.
-
28 Nov 2016 14 repositories listed Syntology 28 ran (of which 0 constructed an object rather than computing a result; 23 with no instrument failure: 0 honoured, 0 violated, 23 with no contract checked; 5 where Syntology's instrument failed) · 5 unverified (of 33 harvested samples)The size of the dataset and the fact that the questions are derived from real user search queries distinguishes MS MARCO from other well-known publicly available datasets for machine reading comprehension and…
-
2 Apr 2019 13 repositories listed Syntology community repositories only · 13 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 1 violated, 10 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 15 harvested samples) · 15 pointer-only (licence)We present Habitat, a platform for research in embodied artificial intelligence (AI).
-
3 Oct 2018 13 repositories listed Syntology official (archive's flag): 4 ran · 11 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 2 honoured, 0 violated, 8 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 11 harvested samples) · 1 pointer-only (licence)Such architectural design and abstractions enable researchers and developers to extend the toolkit with their new algorithms and improvements, and to use it for performance benchmarking.
-
3 Oct 2016 13 repositories listed Syntology community repositories only · 12 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 1 honoured, 0 violated, 10 with no contract checked; 1 where Syntology's instrument failed) · 14 unverified (of 26 harvested samples) · 26 pointer-only (licence)An adversarial example library for constructing attacks, building defenses, and benchmarking both
-
25 Feb 2019 12 repositories listed Syntology 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples)Semantic segmentation of medical images aims to associate a pixel with a label in a medical image without human initialization.
-
29 Mar 2016 12 repositories listedWe introduce COCO, an open source platform for Comparing Continuous Optimizers in a black-box setting.
-
22 Mar 2017 11 repositories listed Syntology official (archive's flag): 1 ran · 25 ran (of which 0 constructed an object rather than computing a result; 18 with no instrument failure: 1 honoured, 0 violated, 17 with no contract checked; 7 where Syntology's instrument failed) · 15 unverified (of 40 harvested samples) · 5 pointer-only (licence)Health care is one of the most exciting frontiers in data mining and machine learning.
-
16 Apr 2022 10 repositories listed Syntology community repositories only · 17 ran (of which 1 constructed an object rather than computing a result; 15 with no instrument failure: 0 honoured, 0 violated, 15 with no contract checked; 2 where Syntology's instrument failed) · 11 unverified (of 28 harvested samples) · 4 pointer-only (licence)This large and diverse collection of tasks enables rigorous benchmarking of cross-task generalization under instructions -- training models to follow instructions on a subset of tasks and evaluating them on the…
-
18 Jul 2018 10 repositories listedSkillful mobile operation in three-dimensional environments is a primary topic of study in Artificial Intelligence.
-
14 Jun 2020 9 repositories listed Syntology official (archive's flag): 3 ran · 11 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 0 violated, 5 with no contract checked; 5 where Syntology's instrument failed) · 1 unverified (of 12 harvested samples) · 7 pointer-only (licence)Multi-agent deep reinforcement learning (MARL) suffers from a lack of commonly-used evaluation tasks and criteria, making comparisons between approaches difficult.
-
22 Oct 2019 9 repositories listed Syntology official (archive's flag): 6 ran · 11 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 1 where Syntology's instrument failed) · 13 unverified (of 24 harvested samples) · 3 pointer-only (licence)Person re-identification (re-ID), which aims to re-identify people across different camera views, has been significantly advanced by deep learning in recent years, particularly with convolutional neural networks (CNNs).
-
13 Mar 2019 9 repositories listedWe have recently seen the emergence of several publicly available Natural Language Understanding (NLU) toolkits, which map user utterances to structured, but more abstract, Dialogue Act (DA) or Intent specifications…
-
11 Apr 2022 8 repositories listedWe also develop a metrics library, ivtmetrics, for model evaluation on surgical triplets.
-
15 Oct 2021 8 repositories listed Syntology official (archive's flag): 6 ran · 13 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 1 violated, 8 with no contract checked; 4 where Syntology's instrument failed) · 2 unverified (of 15 harvested samples) · 3 pointer-only (licence)Large language models have recently been shown to attain reasonable zero-shot generalization on a diverse set of tasks (Brown et al., 2020).
-
9 Aug 2024 7 repositories listedThis document provides an overview of the challenge, including the registration process, rules, submission format, description of the datasets used, qualified team rankings, all team descriptions, and the benchmarking…
-
31 Mar 2019 7 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples) · 4 pointer-only (licence)The morphometry of a kidney tumor revealed by contrast-enhanced Computed Tomography (CT) imaging is an important factor in clinical decision making surrounding the lesion's diagnosis and treatment.
-
3 Dec 2018 7 repositories listed Syntology community repositories only · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)Modern federated networks, such as those comprised of wearable devices, mobile phones, or autonomous vehicles, generate massive amounts of data each day.
-
28 Jan 2022 6 repositories listed Syntology official (archive's flag): 11 ran · 24 ran (of which 0 constructed an object rather than computing a result; 12 with no instrument failure: 2 honoured, 0 violated, 10 with no contract checked; 12 where Syntology's instrument failed) · 5 unverified (of 29 harvested samples) · 7 pointer-only (licence)Deep neural networks on 3D point cloud data have been widely used in the real world, especially in safety-critical applications.
-
12 Sep 2020 6 repositories listed Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)We have publicly released the benchmarking code, evaluation protocols, and hyper-parameter settings of our work to promote reproducible research in this field.
-
5 Nov 2019 6 repositories listed Syntology community repositories only · 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 11 harvested samples)To make deliberate progress towards more intelligent and more human-like artificial systems, we need to be following an appropriate feedback signal: we need to be able to define and evaluate intelligence in a way that…
-
13 Jan 2019 6 repositories listed Syntology 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples)In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the…
-
9 Feb 2024 5 repositories listedMachine-learning from a disparate set of tables, a data lake, requires assembling features by merging and aggregating tables.
-
9 Oct 2023 5 repositories listed Syntology official (archive's flag): 5 ran · 11 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 12 harvested samples) · 6 pointer-only (licence)
-
13 Oct 2022 5 repositories listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)
-
13 Oct 2022 5 repositories listed Syntology community repositories only · 10 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (of 13 harvested samples)
-
26 May 2022 5 repositories listed Syntology official (archive's flag): 1 ran · 11 ran (of which 1 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 13 harvested samples) · 2 pointer-only (licence)
-
19 May 2022 5 repositories listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (of 8 harvested samples) · 1 pointer-only (licence)
-
9 Mar 2022 5 repositories listed Syntology official (archive's flag): 5 ran · 8 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 1 violated, 4 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 9 harvested samples) · 2 pointer-only (licence)
-
29 Jul 2021 5 repositories listed Syntology community repositories only · 8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 8 harvested samples)
-
8 Nov 2020 5 repositories listed Syntology community repositories only · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples)
-
16 Sep 2020 5 repositories listed
-
29 May 2020 5 repositories listed
-
5 May 2020 5 repositories listed Syntology 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
3 Mar 2020 5 repositories listed Syntology official (archive's flag): 4 ran · 14 ran (of which 0 constructed an object rather than computing a result; 14 with no instrument failure: 0 honoured, 1 violated, 13 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 15 harvested samples) · 1 pointer-only (licence)
-
3 Oct 2019 5 repositories listed
-
4 Sep 2019 5 repositories listed
-
27 Aug 2019 5 repositories listed Syntology community repositories only · 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples)
-
7 Mar 2019 5 repositories listed
-
6 Dec 2018 5 repositories listed
-
11 Oct 2018 5 repositories listed
-
8 Oct 2018 5 repositories listed
-
6 Jan 2014 5 repositories listed
-
11 Mar 2025 4 repositories listed Syntology official (archive's flag): 4 ran · 8 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 2 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 9 harvested samples)
-
27 Feb 2025 4 repositories listed
-
24 Jun 2024 4 repositories listed
-
22 Jun 2024 4 repositories listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples) · 1 pointer-only (licence)
-
18 Jun 2024 4 repositories listed
-
6 Jun 2024 4 repositories listed Syntology official (archive's flag): 13 ran · 13 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 10 where Syntology's instrument failed) · 4 unverified (of 17 harvested samples) · 17 pointer-only (licence)
-
9 May 2024 4 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
25 Mar 2024 4 repositories listed
-
12 Mar 2024 4 repositories listed Syntology official (archive's flag): 3 ran · 11 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 2 violated, 6 with no contract checked; 3 where Syntology's instrument failed) · 4 unverified (of 15 harvested samples) · 5 pointer-only (licence)
-
23 Jun 2023 4 repositories listed
-
19 Jun 2023 4 repositories listed Syntology official (archive's flag): 2 ran · 23 ran (of which 0 constructed an object rather than computing a result; 16 with no instrument failure: 1 honoured, 1 violated, 14 with no contract checked; 7 where Syntology's instrument failed) · 2 unverified (of 25 harvested samples) · 9 pointer-only (licence)
-
12 Jun 2023 4 repositories listed Syntology community repositories only · 21 ran (of which 0 constructed an object rather than computing a result; 21 with no instrument failure: 0 honoured, 0 violated, 21 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 22 harvested samples)
-
8 Jun 2023 4 repositories listed Syntology official (archive's flag): 21 ran · 21 ran (of which 0 constructed an object rather than computing a result; 15 with no instrument failure: 1 honoured, 0 violated, 14 with no contract checked; 6 where Syntology's instrument failed) · 4 unverified (of 25 harvested samples) · 1 pointer-only (licence)
-
26 Mar 2023 4 repositories listed Syntology community repositories only · 19 ran (of which 0 constructed an object rather than computing a result; 12 with no instrument failure: 0 honoured, 0 violated, 12 with no contract checked; 7 where Syntology's instrument failed) · 7 unverified (of 26 harvested samples) · 5 pointer-only (licence)
-
13 Oct 2022 4 repositories listed Syntology community repositories only · 15 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 3 honoured, 2 violated, 5 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified (of 17 harvested samples) · 2 pointer-only (licence)
-
20 Sep 2022 4 repositories listed
-
7 Feb 2022 4 repositories listed Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified (of 13 harvested samples) · 2 pointer-only (licence)
-
22 Oct 2021 4 repositories listed
-
2 Aug 2021 4 repositories listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified (of 8 harvested samples)
-
28 Aug 2020 4 repositories listed Syntology community repositories only · 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 2 pointer-only (licence)
-
25 Apr 2020 4 repositories listed
-
31 Aug 2019 4 repositories listed Syntology official (archive's flag): 1 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (of 6 harvested samples) · 3 pointer-only (licence)
-
17 Jul 2019 4 repositories listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples)
-
13 Jun 2019 4 repositories listed Syntology community repositories only · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)
-
25 Feb 2019 4 repositories listed Syntology community repositories only · 9 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 10 harvested samples)
-
13 Nov 2018 4 repositories listed
-
22 Oct 2018 4 repositories listed
-
11 Oct 2018 4 repositories listed Syntology 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
17 Aug 2018 4 repositories listed
-
11 Aug 2017 4 repositories listed
-
24 May 2025 3 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
-
20 May 2025 3 repositories listed
-
9 May 2025 3 repositories listed Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
14 Apr 2025 3 repositories listed
-
16 Jan 2025 3 repositories listed
-
6 Dec 2024 3 repositories listed Syntology official (archive's flag): 15 ran · 17 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 1 violated, 5 with no contract checked; 10 where Syntology's instrument failed) · 7 unverified (of 24 harvested samples) · 24 pointer-only (licence)
-
9 Oct 2024 3 repositories listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 6 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples)
-
2 Oct 2024 3 repositories listed
-
21 Jun 2024 3 repositories listed Syntology community repositories only · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 4 harvested samples)
-
14 Jun 2024 3 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
9 Jun 2024 3 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
6 Jun 2024 3 repositories listed Syntology official (archive's flag): 1 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
4 Jun 2024 3 repositories listed Syntology official (archive's flag): 15 ran · 15 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 1 honoured, 0 violated, 7 with no contract checked; 7 where Syntology's instrument failed) · 3 unverified (of 18 harvested samples)
-
10 May 2024 3 repositories listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 1 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 5 unverified (of 11 harvested samples) · 11 pointer-only (licence)
-
15 Apr 2024 3 repositories listed Syntology official (archive's flag): 13 ran · 13 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 10 where Syntology's instrument failed) · 3 unverified (of 16 harvested samples) · 16 pointer-only (licence)
-
15 Apr 2024 3 repositories listed Syntology official (archive's flag): 3 ran · 7 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 1 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples) · 2 pointer-only (licence)
-
1 Apr 2024 3 repositories listed
-
29 Mar 2024 3 repositories listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples)
Syntology lines on 68 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections