Home › Code › check_correctness

check_correctness

Syntologyentry name in harvested coderead from the graph 2026-09-24

check_correctness appears in the code Syntology harvested for 26 papers, as 24 distinct code bodies found in 26 places (a place is one code body under one paper). At least one of them ran in 8 of the papers; 1 of the code bodies carries a behaviour fingerprint.

What this page is not. Routines are grouped here by the exact string of their function or class name. Nothing asserts that two samples named check_correctness do the same thing, share code, or are comparable; the name is a string, not an identity. Behaviour outputs (what a fingerprinted sample returned on the shared battery) are not in this export and are not shown here; the graph at syntology.ai holds them. "Ran" means executed on a synthesized fixture, not that the code is correct or reproduces a paper.

Samples Syntology

Syntology ran 8 of the 24 distinct code bodies named check_correctness; 16 are unverified. One tile per status, in the site's fixed vocabulary, each code body counted once:

0ran · honoured contract
0ran · violated contract
1ran · our draft was wrong
0ran · fixture could not drive it
7ran
16unverified
1fingerprinted

Licence is a property of each copy, so it is counted per place: 9 of the 26 places are pointer only (Syntology does not serve that copy's text). This site shows no code text for any sample; every row below links to the file in its repository where the record names one.

“Ran” means the sample executed on a synthesized input; it does not mean the output is correct. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself; “our draft was wrong” and “fixture could not drive it” are failures of Syntology's instrument, not of the code, and those samples did run. The ran count above is every status except unverified, the same rule as each paper page.

Papers

26 papers shown of 26, newest first; 26 places in the table. A paper with no recorded date is placed by the month its arXiv id encodes, shown in the Date column as YYYY-MM (from id). One row per place: a paper whose repository defines the name more than once appears more than once, and the same code body held for several papers appears once under each, with the same status. Titles and dates are the archive's archive 2025-07-28 for papers in the archive, and the graph's for 10 papers added by Syntology. Status and fingerprint are Syntology's record of each code body; licence is recorded for each place. The File cell ends with the code body's code_sha256, Syntology's identity for that exact code: an agent fetches the stored sample with Syntology's MCP tool get_code(code_sha256="…") (how to connect); click an identity to copy that call.

PaperDateFileStatus SyntologyLicence
Chopthin-Consensus Power Sampling: A Diversity-Preserving Approach to LLM Decoding added by Syntology 2026-09 (from id) MinooAhmadii/chopthin-consensus-power-sampling/ccps/graders/he_execute.py fe90a22233c11824 unverified licence not identified · pointer only
Codebook Agent: Amortized Topology Design for LLM Multi-Agent Systems added by Syntology 2026-09 (from id) jinxiy1104/CodebookAgent/benchmarks/humaneval_dataset.py a2b71c1db65fa88c unverified no licence file found · pointer only
TARPO: Token-Wise Latent-Explicit Reasoning via Action-Routing Policy Optimization added by Syntology 2026-06 (from id) NKU-LITI/TARPO-master/humanevaleval.py 1177382634e74969 unverified no licence file found · pointer only
Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoning added by Syntology 2026-05 (from id) Usama1002/convergence-without-understanding/src/evaluation.py 42b68d72beb0cfc1 ran MIT (permissive)
Multi-Rollout On-Policy Distillation via Peer Successes and Failures added by Syntology 2026-05 (from id) viviable/mopd_code/verl/utils/reward_score/prime_code/utils.py 2d94e42b971ca757 ran Apache-2.0 (permissive)
CODESPECBENCH: Benchmarking LLMs for Executable Behavioral Specification Generation added by Syntology 2026-04 (from id) SparksofAGI/CodeSpecBench/CodeSpecBench-Func/spec-verifier/evaluation.py 8f623a2fab794092 unverified no licence file found · pointer only
On Representation Redundancy in Large-Scale Instruction Tuning Data Selection added by Syntology 2026-02 (from id) tdano1/CRDS/main/eval/codex_humaneval/execution.py 006afab0fbb96716 unverified Apache-2.0 (permissive)
PACE: Prefix-Protected and Difficulty-Aware Compression for Efficient Reasoning added by Syntology 2026-02 (from id) LiveCodeBench/LiveCodeBench/lcb_runner/evaluation/utils_execute.py d4af0384e850be8d unverified MIT (permissive)
CPMöbius: Iterative Coach-Player Reasoning for Data-Free Reinforcement Learning added by Syntology 2026-02 (from id) thunlp/CPMobius/evaluation/utils/execution.py 12d4fef547070f7c unverified Apache-2.0 (permissive)
When Domains Interact: Asymmetric and Order-Sensitive Cross-Domain Effects in Reinforcement Learning for Reasoning added by Syntology 2026-02 (from id) uservan/cross_domain/verify/score/science.py 8894398260de443f unverified no licence file found · pointer only
Scalable Chain of Thoughts via Elastic Reasoning 8 May 2025 salesforceairesearch/elastic-reasoning/rllm/rewards/code_reward.py 7134e7134bb2121a unverified Apache-2.0 recorded; this copy not marked cleared · pointer only
HumanEval-V: Evaluating Visual Understanding and Reasoning Abilities of Large Multimodal Models Through Coding Tasks 16 Oct 2024 HumanEval-V/HumanEval-V-Benchmark/execution.py f0c3af44403d4d56 ran no licence file found · pointer only
MG-Verilog: Multi-grained Dataset Towards Enhanced LLM-assisted Verilog Generation 2 Jul 2024 luke-avionics/mg-verilog/verilog_eval/verilog_eval/execution.py f327928b4d07faf0 ran MIT (permissive)
Symbolic Learning Enables Self-Evolving Agents 26 Jun 2024 aiwaves-cn/agents/src/agents/datasets/humaneval.py 0ac8c3678ce38802 ran Apache-2.0 (permissive)
TESTEVAL: Benchmarking Large Language Models for Test Case Generation 2024-06 (from id) llm4softwaretesting/testeval/eval_overall.py 14f35d21477c79fd unverified MIT (permissive)
Advancing LLM Reasoning Generalists with Preference Trees 2 Apr 2024 openbmb/eurus/eval/utils/execution.py 12d4fef547070f7c unverified Apache-2.0 (permissive)
HumanEval on Latest GPT Models -- 2024 20 Feb 2024 daniel442li/gpt-human-eval/human_eval/execution.py 8084d416685d852c unverified MIT (permissive)
Scaling Sparse Fine-Tuning to Large Language Models 29 Jan 2024 ducdauge/sft-llm/eval/codex_humaneval/execution.py 12d4fef547070f7c unverified MIT (permissive)
Evaluating LLMs' Mathematical and Coding Competency through Ontology-guided Interventions 17 Jan 2024 declare-lab/llm-reasoningtest/execution.py ba731bb9824c9e75 unverified Apache-2.0 (permissive)
OOP: Object-Oriented Programming Evaluation Benchmark for Large Language Models 12 Jan 2024 alphadl/oop-eval/oop_evaluate/execution.py 669dc622124bea08 unverified no licence file found · pointer only
AST-T5: Structure-Aware Pretraining for Code Generation and Understanding 5 Jan 2024 gonglinyuan/ast_t5/eval/evaluate_mbpp.py 9a1d883bd8569073 ran no licence file found · pointer only
InstructCoder: Instruction Tuning Large Language Models for Code Editing 31 Oct 2023 qishenghu/CodeInstruct/edit_eval/execution.py d76cd6028267121b ran MIT (permissive)
Lemur: Harmonizing Natural Language and Code for Language Agents 10 Oct 2023 OpenLemur/Lemur/xchat/eval/mbpp/execution.py b8cf46548e4cd074 unverified Apache-2.0 (permissive)
CodeT: Code Generation with Generated Tests 21 Jul 2022 microsoft/codet/CodeT/src/_execution.py 4b2cbd2e86925044 ran · our draft was wrong fingerprinted MIT (permissive)
Rethinking Explainability as a Dialogue: A Practitioner's Perspective 3 Feb 2022 dylan-slack/talktomodel/experiments/utils.py b20ffd4960370230 unverified MIT (permissive)
Evaluating Large Language Models Trained on Code 7 Jul 2021 openai/human-eval/human_eval/execution.py 87135198de7ec14b unverified MIT (permissive)

This site shows no code text; each File cell links to the file on GitHub at the repository's current default branch, which may have changed since the harvest. "Pointer only" means Syntology does not serve that copy's text, for one of four reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; or the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar). Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence cell for the reason. Per-sample records for a paper are on its paper page under "Code Syntology ran".

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections