What is not shown

A site that only shows what it has cannot be checked. This page is the build's exclusion ledger: what the archive and the graph hold that no page renders, with the number and the reason. Numbers are from this build; the archive snapshot is 2025-07-28. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record.

The same ledger as JSON, built from the dict this page prints: /not-shown.json.

Pages

WhatCountWhy
papers in the archive catalogue625,187abstracts table plus links-only rows
paper pages rendered225,441papers with a code link, a benchmark row, an introduced dataset or method, or Syntology samples
papers without a page399,746abstract-only: nothing runnable or ranked to show; the search index does not list them
method entries removed as spam (name pattern)12,604customer-service phrasing in the name, counted over the methods table and the paper catalogue's embedded method list; the pattern is public in the build script
duplicate leaderboard tables collapsed10,354the same benchmark reached from several task nodes
method entries removed as spam (dummy-paper clusters)6,785entries hung off five dummy paper records, counted over the methods table and the paper catalogue's embedded method list; see the attribution page
dataset rows removed as spam2,794contact-centre spam in the dataset's name or full name; the pattern is public in the build script
papers excluded for having no title636a page needs a title
per-paper method tags dropped as spam37,766the method-name pattern, applied to the tags the paper catalogue carries
dataset descriptions blanked as vandalised201row kept, text removed
method-collection descriptions blanked as vandalised65text removed
method descriptions blanked as vandalised22row kept, text removed
task descriptions blanked as vandalised5row kept, text removed
per-paper method tags with no method page4,230shown as plain text, not linked
per-paper task tags with no task page29,815not shown; the archive tagged papers with labels that never had a task page
benchmark dataset nodes with no table190no metrics and no rows
leaderboard rows with no paper page3,740row has no paper URL, or a URL that matches no catalogue paper
benchmarks whose dataset has no dataset page3,096name matched nothing in the dataset catalogue
task slugs derived rather than archived1,324the archive's task URLs cover only tasks that a dataset lists
Syntology: paper pages whose arXiv id is not in the Atlas id list108,712checked against the Atlas id list (549,449 ids, as this build read them; the read time is in the build record); these pages link to no Syntology page and do not say the graph lacks the paper, because the list is not the whole graph (7,185 papers in this site's copy of the graph have an arXiv id the list lacks); Syntology's feed shows papers as they enter the graph
Syntology: paper pages that cannot be joined (no arXiv id)12,898this site joins papers to Syntology's graph by arXiv id; these pages do not say whether the graph holds the paper
papers in Syntology's layer with harvested samples and no page here Syntology2,441Of these, 1,954 carry an id in the export that is not an arXiv id (731 ACL Anthology ids, 489 CVF open access ids, 439 AAAI ids, 117 IJCAI ids, 96 OpenReview ids, 82 bare-number ids), and 487 have an arXiv id from before the snapshot that the archive catalogue does not hold. Their 18,061 sample links, and the 2,343 of these papers with a sample that ran, are counted in the site's sample and paper totals. None of them is given a page or a link into Syntology from here; their ids are repaired where the graph layer is exported, not on this site.
  ACL Anthology ids (not arXiv ids)731the graph layer's arxiv_id field holds an id of this kind, not an arXiv id
  CVF open access ids (not arXiv ids)489the graph layer's arxiv_id field holds an id of this kind, not an arXiv id
  AAAI ids (not arXiv ids)439the graph layer's arxiv_id field holds an id of this kind, not an arXiv id
  IJCAI ids (not arXiv ids)117the graph layer's arxiv_id field holds an id of this kind, not an arXiv id
  OpenReview ids (not arXiv ids)96the graph layer's arxiv_id field holds an id of this kind, not an arXiv id
  bare-number ids (not arXiv ids)82the graph layer's arxiv_id field holds an id of this kind, not an arXiv id
  arXiv id from before the snapshot, not in the archive catalogue487neither an archive page nor a page for a paper newer than the archive

Withheld or changed by this site's content rules

Archive rows and fields that the site's own content rules withheld, blanked or changed at build time, each count the rules keep, listed once. Rules: cwp/cwp_lib/data.py PROSE_RULES / NAME_RULES (NFKC-folded text); applied at build time to TEMP copies of the catalog tables, never to the catalog file.

WhatCount
dataset rows withheld, all reasons40
dataset rows withheld: affiliate or product advertising24
dataset rows withheld: adult4
dataset rows withheld: travel-booking or airline-support spam4
dataset rows withheld: gambling advertising4
dataset rows withheld: payment or crypto support2
dataset rows withheld: tech support1
dataset rows withheld: contact-centre spam1
method entries withheld, all reasons32
method entries withheld: contact-centre spam10
method entries withheld: travel-booking or airline-support spam7
method entries withheld: affiliate or product advertising6
method entries withheld: gambling advertising3
method entries withheld: pharmacy advertising2
method entries withheld: a test entry2
method entries withheld: essay mill1
method entries withheld: name and text replaced by an advert (affiliate or product advertising)1
task entries withheld, all reasons2
task entries withheld: contact-centre spam2
leaderboard tables withheld, all reasons27
leaderboard tables withheld: its only row was a test entry23
leaderboard tables withheld: every row was withheld as spam3
leaderboard tables withheld: its task was withheld1
leaderboard rows withheld, all reasons26
leaderboard rows withheld: a single row credited to a dummy paper record23
leaderboard rows withheld: contact-centre spam2
leaderboard rows withheld: travel-booking or airline-support spam1
dataset rows: description blanked, row kept: llm paste35
method entries: description blanked, row kept: one junk word as the whole text2
dataset rows: licence link blanked, row kept: it pointed at a spam or darknet-market host1
dataset rows: description blanked, row kept: darknet advertising1
dataset rows: licence name blanked, row kept: it repeated the spam name that replaced the dataset's own name in the same edit (the name was restored from the dataset's address; the licence link was removed with it)1
method entries: description blanked, row kept: a phone number1
task entries: description blanked, row kept: a phone number1
dataset names changed in the same edit as a spam description, restored from the dataset's address2
distinct method tag names on papers withheld as spam255
distinct task tag names on papers withheld as spam1
method "see code" links shown as plain text: not a code host26
method source links shown as plain text: not a paper host12

What the archive cannot supply at all

Results Syntology placed on archive leaderboards Syntology

Syntology has checked 8,886 of the 18,637 papers on this site that are newer than the archive (for 3,717 of them no archive leaderboard matched the paper's tables, so there was nothing further to check); 733 were read and have nothing to place (arXiv has no HTML version of the paper, or that version has no tables), 41 could not be read (the extractor's reply could not be parsed), and results from the other 8,977 appear after they are checked.

Every paper the leaderboard extractor read, and what became of each leaderboard it proposed. A model pointed at the cell in the paper's own table; the number was read from that cell and checked against the leaderboard's metric, dataset, split and scale; an independent check that saw the leaderboard's other rows and every other leaderboard on the same dataset accepted or rejected the entry. Only accepted entries are shown anywhere on the site.

WhatCountWhy
papers on this site newer than the archive18,637the extractor reads every one, and the independent check follows wherever it proposes a leaderboard; counted from this build's graph layer
checked8,886the extractor read the paper's tables and compared them with the archive's leaderboards, and the independent check gave a verdict (accept or reject) on every leaderboard the extractor proposed; for 3,717 of these papers no archive leaderboard matched the tables, so none was proposed; a paper checked in part can also show a result, on the leaderboards the check accepted
read, nothing to place733arXiv has no HTML version of the paper (260), or that version has no tables (473); there is no table to place a result from, so these are not counted as checked
could not be read41the extractor's reply could not be parsed; not counted as checked, and nothing from these papers is shown
read by the extractor, not yet checked0nothing from these papers is shown until the check has run
checked in part2the check gave no verdict on at least one of the paper's leaderboards; nothing unchecked is shown
not yet read by the extractor8,975the extractor has not reached the paper yet, or its last attempt stopped before writing a result and the paper is read again on the next run
leaderboards awaiting a verdict3(paper, leaderboard) pairs with rule-passing cells the independent check has not yet accepted or rejected; none is shown
papers with a line in the verified file9,662one line per paper the extractor finished with, whatever the outcome, read once at build time (29,998,752 bytes, sha256 e64d2b4c9755…); by outcome below
papers: answered5,171the extractor answered for the paper's tables
papers: no candidate board3,717no archive leaderboard plausibly matches the paper's tables
papers: no tables473the paper's HTML rendering has no tables
papers: no html260arXiv holds no HTML rendering of the paper, so it has no readable tables
papers: unparseable41the extractor's reply could not be parsed; counted as could not be read, never as checked
leaderboards proposed4,465(paper, leaderboard) pairs the model said report the paper's own method on that leaderboard
refusals by a rule9,670each cell, row or configuration the model pointed at that failed a deterministic check, counted once under the first check it failed (below); none is shown
rule: metric not in column header3,803the table's column header does not name the leaderboard's metric
rule: cell text mismatch1,224the number the model quoted is not the number printed in the cell it pointed at
rule: dataset not in table849the table does not name the leaderboard's dataset
rule: scale mismatch749the value is not on the scale of the leaderboard column's archive values
rule: no quoted value648the model gave no value to check against the cell
rule: label mismatch628the row label the model quoted is not the row's label
rule: ablation label547the row is an ablation variant, not the method itself
rule: one configuration two values273one configuration gave two values for one column
rule: split mismatch265the table reports a different split or setting than the leaderboard
rule: header row206the model pointed at a header row
rule: several configurations no direction157several configurations of the method, and no metric direction to pick one
rule: quoted value ambiguous in row141the quoted value appears more than once in the row
rule: metric qualifier in header70the column header qualifies the metric in a way the leaderboard does not
rule: board column not on board36the column named is not one of the leaderboard's columns
rule: bad coordinates34the table, row or column pointed at does not exist
rule: header names two metrics27the column header names two metrics
rule: not a configuration row13the row is not a configuration of the method
independent check: reject969rejected by the independent check
independent check: accept132accepted by the independent check; placed on the leaderboard
independent check: unparseable3the independent check's reply could not be parsed; not placed
entries placed on leaderboards132one per paper per leaderboard; only these render
cells placed160the numbers those entries print, one per leaderboard column
leaderboards carrying an entry95listed below the archive rows, never ranked among them
papers with an entry93each has a paper page that says where

Every leaderboard carrying a placed result, with its number of entries and newest paper: leaderboards with results placed since the archive.

Measured before publishing, on held-out archive papers: the extractor was given 883 of them and produced output for 881 (the other 2 had no title and were excluded before any model call, so they are not part of this measurement). A model adjudicator (Claude Opus 5.5), blind to which entries were real, judged 108 of 110 accepted entries correct on those held-out archive papers; its own controls: 69 of 70 archive-confirmed entries judged true, 101 of 103 corrupted copies judged false. It was not measured on the newer papers whose results are placed on this site. The rules were frozen before that test was run.

Papers newer than the archive

WhatCountWhy
post-freeze paper pages18,637a repository link in Syntology's graph, or (0 of them) no repository but an extracted result attached to an archive table
with arXiv's metadata (title, abstract, authors, date)18,637arXiv metadata, CC0 1.0
without arXiv's metadata0arXiv's metadata file holds no record for them yet; the page shows the graph's title and says authors are on arXiv

Changed after a reader report

No decided report changed anything in this build.

Syntology's layer, in numbers

Two units, named on every row. A sample is one distinct piece of code, counted once however many papers' repositories hold it; these are the same counts Code prints. A link is one sample under one paper, so identical code held for several papers is one sample and several links. Licence and where a copy came from are properties of the link; status and fingerprint are properties of the sample.

WhatCountUnit
papers in Syntology's layer63,466papers
papers with harvested code samples51,887papers
papers newer than the archive with a repository link in the graph18,637papers
distinct code samples294,260samples
samples that ran (any status except unverified)211,169samples
of which checked (honoured, violated, or ran with no contract)147,384samples
of which Syntology's instrument failed (our draft was wrong, or the contract check could not run): failures of Syntology's instrument, not of the code63,785samples
ran, honoured the contract we drafted15,745samples
ran, violated the contract we drafted5,703samples
ran; our contract draft was wrong, not the code50,332samples
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)13,453samples
ran on a synthesized input125,936samples
unverified (harvested, no recorded run)83,091samples
samples with a behaviour fingerprint58,064samples
links whose sample has no function or class name0links
paper–sample links361,576links
links that are pointer only (Syntology does not serve that copy's text)142,682links
of which this paper's copy is cleared, but the stored copy of the same code that Syntology serves is not17,699links
links where this paper's copy is the one Syntology first harvested243,453links
links whose repository and path the harvester recorded for this paper100,333links
links whose repository and path come from a harvest plan row for this paper12,948links
links to identical code first harvested for another paper (this paper's copy not located)4,842links

A sample is never promoted from "unverified" without a record: harvesting is cheap and execution is not. Statuses are computed at build time by the site's one mapping from the recorded level, report and contract verdict; the downloadable file carries the same statuses (download).