Browse State-of-the-Art › Classifier calibration
Classifier calibration
19 papers with code · 1 benchmark · 1 dataset archive 2025-07-28
Confidence calibration – the problem of predicting probability estimates representative of the true correctness likelihood – is important for classification models in many applications. The two common calibration metrics are Expected Calibration Error (ECE) and Maximum Calibration Error (MCE).
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| CIFAR-100 (1 row) | R-Mix (PreActResNet-18) | Expeditious Saliency-guided Mix-up through Random Gradient Thresholding | code | — | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
1 dataset whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 29 · with a code link: 19 · where Syntology ran a sample: 6 (4 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (6 of 29 tagged: 4 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 29 of 29, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from.
-
15 Dec 2020 4 repositories listedOur central intuition is that there is a continuous spectrum of ensemble-like models of which MC-Dropout and Deep Ensembles are extreme examples.
-
9 Jun 2021 2 repositories listed Syntology 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result (of 1 harvested sample) · 1 pointer-only (licence)Motivated by the above findings, we propose a novel and simple algorithm called Classifier Calibration with Virtual Representations (CCVR), which adjusts the classifier using virtual representations sampled from an…
-
13 May 2025 1 repository listedClass Incremental Learning (CIL) based on pre-trained models offers a promising direction for open-world continual learning.
-
5 Feb 2025 1 repository listedExisting works formulated this challenge as a long-tailed problem and attempted to tackle it by decoupling the feature representation and classification.
-
20 Dec 2024 1 repository listedFurthermore, existing GFSS approaches suffer from a lack of contextual information for novel classes due to their limited samples, we thereby introduce a context consistency learning scheme to transfer the contextual…
-
29 Sep 2024 1 repository listedOur thorough evaluation and comparison of different calibration methods have shown improved accuracy in user identification across multiple datasets.
-
21 Feb 2024 1 repository listedWe propose an accuracy-preserving calibration method using the Concrete distribution as the probabilistic model on the probability simplex.
-
17 Mar 2023 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)Recent advances in neural collapse have shown that the classifiers and feature prototypes under perfect training scenarios collapse into an optimal structure called simplex equiangular tight frame (ETF).
-
9 Dec 2022 1 repository listedMix-up training approaches have proven to be effective in improving the generalization ability of Deep Neural Networks.
-
17 Nov 2022 1 repository listedThis enables client models to be updated in a shared feature space with consistent classifiers during local training.
-
17 Oct 2022 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples)Deep Ensembles (DE) are a prominent approach for achieving excellent performance on key metrics such as accuracy, calibration, uncertainty estimation, and out-of-distribution detection.
-
7 Oct 2022 1 repository listedWe prove for several notions of calibration that solving the reduced problem minimizes the corresponding notion of miscalibration in the full problem, allowing the use of non-parametric recalibration methods that fail…
-
3 Feb 2022 1 repository listedHowever, these methods are unable to detect subpopulations where calibration could also improve prediction accuracy.
-
18 Aug 2021 1 repository listed Syntology official (archive's flag): 10 ran · 10 ran (of which 3 constructed an object rather than computing a result; 7 with no instrument failure: 3 honoured, 0 violated, 4 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (of 13 harvested samples) · 1 pointer-only (licence)Both generalized and incremental few-shot learning have to deal with three major challenges: learning novel classes from only few samples per class, preventing catastrophic forgetting of base classes, and classifier…
-
18 Mar 2021 1 repository listedInterestingly, ViT achieves results superior to CNN baselines with 80.
-
9 Feb 2021 1 repository listedA calibrator is a function that maps the arbitrary classifier score, of a testing observation, onto [0, 1] to provide an estimate for the posterior probability of belonging to one of the two classes.
-
26 Nov 2020 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples)We evaluate the transfer performance of 13 top self-supervised models on 40 downstream tasks, including many-shot and few-shot recognition, object detection, and dense prediction.
-
28 Apr 2020 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples)Therefore, we present a novel framework to measure and calibrate biased (or miscalibrated) confidence estimates of object detection methods.
-
16 Jun 2017 1 repository listedInductive (IVAP) and cross (CVAP) Venn–Abers predictors are computationally efficient algorithms for probabilistic prediction in binary classification problems.
-
FedSA: A Unified Representation Learning via Semantic Anchors for Prototype-based Federated Learning9 Jan 2025 0 repositories listedPrototype-based federated learning has emerged as a promising approach that shares lightweight prototypes to transfer knowledge among clients with data heterogeneity in a model-agnostic manner.
-
10 Jan 2024 0 repositories listedMachine learning models typically focus on specific targets like creating classifiers, often based on known population feature distributions in a business context.
-
Classifier Calibration with ROC-Regularized Isotonic Regression21 Nov 2023 0 repositories listedCalibration of machine learning classifiers is necessary to obtain reliable and interpretable predictions, bridging the gap between model confidence and actual probabilities.
-
What is Your Metric Telling You? Evaluating Classifier Calibration under Context-Specific Definitions of Reliability23 May 2022 0 repositories listedClassifier calibration has received recent attention from the machine learning community due both to its practical utility in facilitating decision making, as well as the observation that modern neural network…
-
Classifier Calibration: A survey on how to assess and improve predicted class probabilities20 Dec 2021 0 repositories listedThis paper provides both an introduction to and a detailed overview of the principles and practice of classifier calibration.
-
Better Classifier Calibration for Small Data Sets24 Feb 2020 0 repositories listedClassifier calibration does not always go hand in hand with the classifier's ability to separate the classes.
-
High Frequency Residual Learning for Multi-Scale Image Classification7 May 2019 0 repositories listedWe present a novel high frequency residual learning framework, which leads to a highly efficient multi-scale network (MSNet) architecture for mobile and embedded vision problems.
-
Binary Classifier Calibration using an Ensemble of Near Isotonic Regression Models16 Nov 2015 0 repositories listedLearning accurate probabilistic models from data is crucial in many practical tasks in data mining.
-
Binary Classifier Calibration: Non-parametric approach14 Jan 2014 0 repositories listedAccurate calibration of probabilistic predictive models learned is critical for many practical prediction and decision-making tasks.
-
Binary Classifier Calibration: Bayesian Non-Parametric Approach13 Jan 2014 0 repositories listedA set of probabilistic predictions is well calibrated if the events that are predicted to occur with probability p do in fact occur about p fraction of the time.
Syntology lines on 6 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections