{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/improving-uncertainty-calibration-of-deep","title":"Improving Uncertainty Calibration of Deep Neural Networks via Truth Discovery and Geometric Optimization","arxiv_id":"2106.14662","date":"2021-06-25","proceeding":null,"authors":["Chunwei Ma","Ziyun Huang","Jiayi Xian","Mingchen Gao","Jinhui Xu"],"abstract":"Deep Neural Networks (DNNs), despite their tremendous success in recent years, could still cast doubts on their predictions due to the intrinsic uncertainty associated with their learning process. Ensemble techniques and post-hoc calibrations are two types of approaches that have individually shown promise in improving the uncertainty calibration of DNNs. However, the synergistic effect of the two types of methods has not been well explored. In this paper, we propose a truth discovery framework to integrate ensemble-based and post-hoc calibration methods. Using the geometric variance of the ensemble candidates as a good indicator for sample uncertainty, we design an accuracy-preserving truth estimator with provably no accuracy drop. Furthermore, we show that post-hoc calibration can also be enhanced by truth discovery-regularized optimization. On large-scale datasets including CIFAR and ImageNet, our method shows consistent improvement against state-of-the-art calibration approaches on both histogram-based and kernel density-based evaluation metrics. Our codes are available at https://github.com/horsepurve/truly-uncertain.","url_abs":"https://arxiv.org/abs/2106.14662v3","url_pdf":"https://arxiv.org/pdf/2106.14662v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"improving-uncertainty-calibration-of-deep","repo_url":"https://github.com/horsepurve/truly-uncertain","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2106.14662","atlas_url":"https://app.syntology.ai/?focus=2106.14662","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.14662"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/horsepurve/truly-uncertain","reach":null}],"summary":{"ran_violates":1,"ran_honours":1,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0c375edf64d3af34","entry":"see_v_s","repo":"horsepurve/truly-uncertain","repo_kind":"official","path":"analysis/UN_utils.py","file_url":"https://github.com/horsepurve/truly-uncertain/blob/HEAD/analysis/UN_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0c375edf64d3af34"}},{"code_sha256_prefix":"5edfcd162422a24c","entry":"w_l","repo":"horsepurve/truly-uncertain","repo_kind":"official","path":"analysis/UN_utils.py","file_url":"https://github.com/horsepurve/truly-uncertain/blob/HEAD/analysis/UN_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5edfcd162422a24c"}},{"code_sha256_prefix":"c01f170a1e1faee6","entry":"see_L","repo":"horsepurve/truly-uncertain","repo_kind":"official","path":"analysis/UN_utils.py","file_url":"https://github.com/horsepurve/truly-uncertain/blob/HEAD/analysis/UN_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c01f170a1e1faee6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}