</> Code with Papers by Syntology

Browse State-of-the-Art › HumanEval › Papers where code ran, page 1

HumanEval

Papers archive 2025-07-28

archive papers tagged: 264 · with a code link: 135 · where Syntology ran a sample: 76 (65 with a run with no instrument failure, 11 where every run was a failure of Syntology's instrument) Syntology

Show: all tagged papersonly where code ran (76 of 264 tagged: 65 with a run with no instrument failure, 11 where every run was a failure of Syntology's instrument)

Syntology We ran code from the paper's repository; we did not run it on this task or check it against the task's benchmarks.

Page 1 of 1: papers 1 to 76 of the 76 tagged papers where Syntology ran at least one harvested sample (65 with a run with no instrument failure, 11 where every run was a failure of Syntology's instrument), newest first by the archive's date (ties by arXiv id). This is a filter on Syntology's record ordered by date only, not a ranking; a run is not a correctness claim. A paper missing from this list is not a recorded non-run: it may have no arXiv id, no harvested code, or only samples that have not run yet.

Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. Abstracts are on each paper's page.

Syntology lines on 76 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page has not changed since , the first build that kept a record date for it; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.