Browse State-of-the-Art › Holdout Set
Holdout Set
17 papers with code · 1 benchmark · 1 dataset archive 2025-07-28
Benchmarks archive 2025-07-28
1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| xView3-SAR (5 rows) | BloodAxe, 1st place xView3 prize challenge | xView3-SAR: Detecting Dark Fishing Activity Using Synthetic... | code | — | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
1 dataset whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 35 · with a code link: 17 · where Syntology ran a sample: 3 (3 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (3 of 35 tagged: 3 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 35 of 35, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
7 Jan 2021 3 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 3 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)While improving prediction accuracy has been the focus of machine learning in recent years, this alone does not suffice for reliable decision-making.
-
30 Jun 2024 2 repositories listedTransformer based large-language models (LLMs) display extreme proficiency with language yet a precise understanding of how they work remains elusive.
-
5 Feb 2025 1 repository listedIn this work, we empirically find that the tuning bias - the coverage gap introduced by leveraging the same dataset for tuning and calibration, is negligible for simple parameter tuning in many conformal prediction…
-
13 Dec 2024 1 repository listedHowever, existing methods of improving multicalibration involve reducing initial training data to create a holdout set for post-processing, which is not ideal when minority training data is already sparse.
-
12 Aug 2024 1 repository listedIn total, our combined dataset of 1.
-
31 May 2024 1 repository listedObjectives: To present a publicly available torso segmentation network for large epidemiology datasets on volumetric interpolated breath-hold examination (VIBE) images.
-
17 Jul 2023 1 repository listedLiver tumor segmentation and classification are important tasks in computer aided diagnosis.
-
9 May 2023 1 repository listedHigh complexity models are notorious in machine learning for overfitting, a phenomenon in which models well represent data but fail to generalize an underlying data generating process.
-
2 Jun 2022 1 repository listedUnsustainable fishing practices worldwide pose a major threat to marine resources and ecosystems.
-
13 Feb 2022 1 repository listedTo address this, we propose updating using a `holdout set' - a subset of the population that does not receive interventions guided by the risk score.
-
31 May 2021 1 repository listedThe training of neural networks is usually monitored with a validation (holdout) set to estimate the generalization of the model.
-
1 May 2021 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 8 unverified (of 16 harvested samples) · 2 pointer-only (licence)To assess generalization, machine learning scientists typically either (i) bound the generalization gap and then (after training) plug in the empirical risk to obtain a bound on the true risk; or (ii) validate…
-
29 Oct 2019 1 repository listedIn this paper, we aim to develop a comprehensive methodology to test different kinds of memory in an agent and assess how well the agent can apply what it learns in training to a holdout set that differs from the…
-
18 Apr 2019 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples)The subsequent LRP visualization revealed that the CNN model focuses indeed on individual lesions, but also incorporates additional information such as lesion location, non-lesional white matter or gray matter areas…
-
4 Apr 2019 1 repository listedAutomatic search of neural architectures for various vision and natural language tasks is becoming a prominent tool as it allows to discover high-performing structures on any dataset of interest.
-
27 Feb 2018 1 repository listedOur aim was to evaluate a new method that could partially automate the identification of trial registrations that may be relevant for systematic review updates.
-
8 Jun 2015 1 repository listedWe also formalize and address the general problem of data reuse in adaptive data analysis.
-
Outcome-based Reinforcement Learning to Predict the Future23 May 2025 0 repositories listedReinforcement learning with verifiable rewards (RLVR) has boosted math and coding in large language models, yet there has been little effort to extend RLVR into messier, real-world domains like forecasting.
-
The DCR Delusion: Measuring the Privacy Risk of Synthetic Data2 May 2025 0 repositories listedSynthetic data has become an increasingly popular way to share data without revealing sensitive information.
-
Navigating Towards Fairness with Data Selection15 Dec 2024 0 repositories listedMachine learning algorithms often struggle to eliminate inherent data biases, particularly those arising from unreliable labels, which poses a significant challenge in ensuring fairness.
-
Benchmark Inflation: Revealing LLM Performance Gaps Using Retro-Holdouts11 Oct 2024 0 repositories listedThe training data for many Large Language Models (LLMs) is contaminated with test data.
-
STAND: Data-Efficient and Self-Aware Precondition Induction for Interactive Task Learning11 Sep 2024 0 repositories listedSTAND is a data-efficient and computationally efficient machine learning approach that produces better classification accuracy than popular approaches like XGBoost on small-data tabular classification problems like…
-
Machine Learning for Quantifier Selection in cvc526 Aug 2024 0 repositories listedIn this work we considerably improve the state-of-the-art SMT solving on first-order quantified problems by efficient machine learning guidance of quantifier selection.
-
The Benefits and Risks of Transductive Approaches for AI Fairness17 Jun 2024 0 repositories listedRecently, transductive learning methods, which leverage holdout sets during training, have gained popularity for their potential to improve speed, accuracy, and fairness in machine learning models.
-
Diversified Ensembling: An Experiment in Crowdsourced Machine Learning16 Feb 2024 0 repositories listedCrowdsourced machine learning on competition platforms such as Kaggle is a popular and often effective method for generating accurate models.
-
Morphological Change Forecasting for Prostate Glands using Feature-based Registration and Kernel Density Extrapolation16 Jan 2021 0 repositories listedOrgan morphology is a key indicator for prostate disease diagnosis and prognosis.
-
Application of Deep Neural Networks to assess corporate Credit Rating4 Mar 2020 0 repositories listedRecent literature implements machine learning techniques to assess corporate credit rating based on financial statement reports.
-
Who Wins the Game of Thrones? How Sentiments Improve the Prediction of Candidate Choice29 Feb 2020 0 repositories listedThis paper analyzes how candidate choice prediction improves by different psychological predictors.
-
A Meta-Analysis of Overfitting in Machine Learning1 Dec 2019 0 repositories listedWe conduct the first large meta-analysis of overfitting due to test set reuse in the machine learning community.
-
Adaptive Statistical Learning with Bayesian Differential Privacy2 Nov 2019 0 repositories listedIn statistical learning, a dataset is often partitioned into two parts: the training set and the holdout (i.e., testing) set.
-
Predicting Individual Responses to Vasoactive Medications in Children with Septic Shock15 Jan 2019 0 repositories listed
-
The Generic Holdout: Preventing False-Discoveries in Adaptive Data Science14 Sep 2018 0 repositories listed
-
Using Poisson Binomial GLMs to Reveal Voter Preferences4 Feb 2018 0 repositories listed
-
Malaria Likelihood Prediction By Effectively Surveying Households Using Deep Reinforcement Learning25 Nov 2017 0 repositories listed
-
Challenges in Bayesian Adaptive Data Analysis8 Apr 2016 0 repositories listed
Syntology lines on 3 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections