{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/uncertainty-estimates-of-predictions-via-a","title":"Uncertainty Estimates of Predictions via a General Bias-Variance Decomposition","arxiv_id":"2210.12256","date":"2022-10-21","proceeding":null,"authors":["Sebastian G. Gruber","Florian Buettner"],"abstract":"Reliably estimating the uncertainty of a prediction throughout the model lifecycle is crucial in many safety-critical applications. The most common way to measure this uncertainty is via the predicted confidence. While this tends to work well for in-domain samples, these estimates are unreliable under domain drift and restricted to classification. Alternatively, proper scores can be used for most predictive tasks but a bias-variance decomposition for model uncertainty does not exist in the current literature. In this work we introduce a general bias-variance decomposition for proper scores, giving rise to the Bregman Information as the variance term. We discover how exponential families and the classification log-likelihood are special cases and provide novel formulations. Surprisingly, we can express the classification case purely in the logit space. We showcase the practical relevance of this decomposition on several downstream tasks, including model ensembles and confidence regions. Further, we demonstrate how different approximations of the instance-level Bregman Information allow reliable out-of-distribution detection for all degrees of domain drift.","url_abs":"https://arxiv.org/abs/2210.12256v3","url_pdf":"https://arxiv.org/pdf/2210.12256v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"uncertainty-estimates-of-predictions-via-a","repo_url":"https://github.com/mlo-lab/uncertainty_estimates_via_bvd","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"out-of-distribution-detection","task_name":"Out-of-Distribution Detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2210.12256","atlas_url":"https://app.syntology.ai/?focus=2210.12256","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.12256"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mlo-lab/uncertainty_estimates_via_bvd","reach":null}],"summary":{"ran_fixture":1,"ran_honours":2},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"aac24439c4a57c8d","entry":"BI_LSE","repo":"mlo-lab/uncertainty_estimates_via_bvd","repo_kind":"official","path":"torch_utils.py","file_url":"https://github.com/mlo-lab/uncertainty_estimates_via_bvd/blob/HEAD/torch_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aac24439c4a57c8d"}},{"code_sha256_prefix":"77005db55fd5f098","entry":"d_LSE","repo":"mlo-lab/uncertainty_estimates_via_bvd","repo_kind":"official","path":"torch_utils.py","file_url":"https://github.com/mlo-lab/uncertainty_estimates_via_bvd/blob/HEAD/torch_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"77005db55fd5f098"}},{"code_sha256_prefix":"d90fe01f0815d0a6","entry":"inner_product","repo":"mlo-lab/uncertainty_estimates_via_bvd","repo_kind":"official","path":"torch_utils.py","file_url":"https://github.com/mlo-lab/uncertainty_estimates_via_bvd/blob/HEAD/torch_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d90fe01f0815d0a6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}