{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/calibrated-prediction-intervals-for-neural","title":"Calibrated Prediction Intervals for Neural Network Regressors","arxiv_id":"1803.09546","date":"2018-03-26","proceeding":null,"authors":["Gil Keren","NIcholas Cummins","Björn Schuller"],"abstract":"Ongoing developments in neural network models are continually advancing the\nstate of the art in terms of system accuracy. However, the predicted labels\nshould not be regarded as the only core output; also important is a\nwell-calibrated estimate of the prediction uncertainty. Such estimates and\ntheir calibration are critical in many practical applications. Despite their\nobvious aforementioned advantage in relation to accuracy, contemporary neural\nnetworks can, generally, be regarded as poorly calibrated and as such do not\nproduce reliable output probability estimates. Further, while post-processing\ncalibration solutions can be found in the relevant literature, these tend to be\nfor systems performing classification. In this regard, we herein present two\nnovel methods for acquiring calibrated predictions intervals for neural network\nregressors: empirical calibration and temperature scaling. In experiments using\ndifferent regression tasks from the audio and computer vision domains, we find\nthat both our proposed methods are indeed capable of producing calibrated\nprediction intervals for neural network regressors with any desired confidence\nlevel, a finding that is consistent across all datasets and neural network\narchitectures we experimented with. In addition, we derive an additional\npractical recommendation for producing more accurate calibrated prediction\nintervals. We release the source code implementing our proposed methods for\ncomputing calibrated predicted intervals. The code for computing calibrated\npredicted intervals is publicly available.","url_abs":"http://arxiv.org/abs/1803.09546v3","url_pdf":"http://arxiv.org/pdf/1803.09546v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"calibrated-prediction-intervals-for-neural","repo_url":"https://github.com/cruvadom/Prediction_Intervals","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"prediction-intervals","task_name":"Prediction Intervals"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}