Post-hoc Calibration

Post-hoc Calibration Dataset

This dataset collection is designed for the evaluation and development of post-hoc calibration methods for deep neural network classifiers. It provides precomputed logits and labels across a range of standard classification benchmarks, enabling rigorous, reproducible calibration research.

🧾 Dataset Characteristics

• Contains logits and ground-truth labels from pretrained neural networks on popular datasets: CIFAR-10, CIFAR-100, SVHN, Stanford Cars (CARS), CUB-200 (BIRDS), and ImageNet.
• Datasets are split into training and test sets, tailored specifically for post-hoc calibration tasks.
• Pretrained networks include ResNet, WideResNet, DenseNet, Swin Transformer, and task-specific fine-tuned architectures.

🎯 Motivation and Content Summary

Modern neural networks often produce poorly calibrated probability estimates, which can hinder decision-making in risk-sensitive applications. This dataset addresses the need for standardized benchmarks for post-hoc calibration by offering a unified collection of: - Ground-truth labels and classification results (logits) from diverse architectures - Calibration tasks across datasets with varying number of classes and granularity - Consistent experimental setup for fair comparison across different calibration methods

💡 Potential Use Cases

• Developing and benchmarking post-hoc calibration algorithms
• Evaluating the generalization ability of calibration methods across domains and data distributions
• Supporting studies in neural network confidence estimation and uncertainty quantification

For more information, refer to the paper "h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objective" (TPAMI 2025) and the official GitHub repository.