{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/locally-valid-and-discriminative-confidence","title":"Locally Valid and Discriminative Prediction Intervals for Deep Learning Models","arxiv_id":"2106.00225","date":"2021-06-01","proceeding":"NeurIPS 2021 12","authors":["Zhen Lin","Shubhendu Trivedi","Jimeng Sun"],"abstract":"Crucial for building trust in deep learning models for critical real-world applications is efficient and theoretically sound uncertainty quantification, a task that continues to be challenging. Useful uncertainty information is expected to have two key properties: It should be valid (guaranteeing coverage) and discriminative (more uncertain when the expected risk is high). Moreover, when combined with deep learning (DL) methods, it should be scalable and affect the DL model performance minimally. Most existing Bayesian methods lack frequentist coverage guarantees and usually affect model performance. The few available frequentist methods are rarely discriminative and/or violate coverage guarantees due to unrealistic assumptions. Moreover, many methods are expensive or require substantial modifications to the base neural network. Building upon recent advances in conformal prediction [13, 33] and leveraging the classical idea of kernel regression, we propose Locally Valid and Discriminative prediction intervals (LVD), a simple, efficient, and lightweight method to construct discriminative prediction intervals (PIs) for almost any DL model. With no assumptions on the data distribution, such PIs also offer finite-sample local coverage guarantees (contrasted to the simpler marginal coverage). We empirically verify, using diverse datasets, that besides being the only locally valid method for DL, LVD also exceeds or matches the performance (including coverage rate and prediction accuracy) of existing uncertainty quantification methods, while offering additional benefits in scalability and flexibility.","url_abs":"https://arxiv.org/abs/2106.00225v4","url_pdf":"https://arxiv.org/pdf/2106.00225v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"locally-valid-and-discriminative-confidence","repo_url":"https://github.com/zlin7/lvd","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"conformal-prediction","task_name":"Conformal Prediction"},{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"prediction-intervals","task_name":"Prediction Intervals"},{"task_slug":"uncertainty-quantification","task_name":"Uncertainty Quantification"},{"task_slug":null,"task_name":"valid"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.00225","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.00225"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/zlin7/LVD","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zlin7/lvd","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"ran_fixture":2,"unverified":12},"by_repo_kind":{"official":{"samples":16,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d6e7f479334eeec3","entry":"NaiveKernel","repo":"zlin7/lvd","repo_kind":"official","path":"models/conformal.py","file_url":"https://github.com/zlin7/lvd/blob/HEAD/models/conformal.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d6e7f479334eeec3"}},{"code_sha256_prefix":"93ae6a4d87ad6dc2","entry":"gpuid_to_device","repo":"zlin7/lvd","repo_kind":"official","path":"models/conformal.py","file_url":"https://github.com/zlin7/lvd/blob/HEAD/models/conformal.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"93ae6a4d87ad6dc2"}},{"code_sha256_prefix":"dad8377818739ce0","entry":"merge_dict_inline","repo":"zlin7/lvd","repo_kind":"official","path":"models/conformal.py","file_url":"https://github.com/zlin7/lvd/blob/HEAD/models/conformal.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dad8377818739ce0"}},{"code_sha256_prefix":"9c7355cbbe4b3419","entry":"weighted_quantile_faster2","repo":"zlin7/lvd","repo_kind":"official","path":"models/conformal.py","file_url":"https://github.com/zlin7/lvd/blob/HEAD/models/conformal.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9c7355cbbe4b3419"}},{"code_sha256_prefix":"7e2b26ed7dad164c","entry":"CQR_from_results","repo":"zlin7/LVD","repo_kind":"official","path":"models/baselines/CQR.py","file_url":"https://github.com/zlin7/LVD/blob/HEAD/models/baselines/CQR.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7e2b26ed7dad164c"}},{"code_sha256_prefix":"a347cfbd81f921df","entry":"LocalConditional","repo":"zlin7/lvd","repo_kind":"official","path":"models/conformal.py","file_url":"https://github.com/zlin7/lvd/blob/HEAD/models/conformal.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a347cfbd81f921df"}},{"code_sha256_prefix":"d3eb64fc08ffefb5","entry":"MADSplit_from_results","repo":"zlin7/LVD","repo_kind":"official","path":"models/baselines/MADSplit.py","file_url":"https://github.com/zlin7/LVD/blob/HEAD/models/baselines/MADSplit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d3eb64fc08ffefb5"}},{"code_sha256_prefix":"077638700e3448f0","entry":"PIConstructor","repo":"zlin7/lvd","repo_kind":"official","path":"models/conformal.py","file_url":"https://github.com/zlin7/lvd/blob/HEAD/models/conformal.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"077638700e3448f0"}},{"code_sha256_prefix":"9199fc21b3ba4e69","entry":"build_architecture","repo":"zlin7/LVD","repo_kind":"official","path":"models/baselines/DNN.py","file_url":"https://github.com/zlin7/LVD/blob/HEAD/models/baselines/DNN.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9199fc21b3ba4e69"}},{"code_sha256_prefix":"38a69e35d2a391c7","entry":"clean_df","repo":"zlin7/LVD","repo_kind":"official","path":"utils/eval_utils.py","file_url":"https://github.com/zlin7/LVD/blob/HEAD/utils/eval_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"38a69e35d2a391c7"}},{"code_sha256_prefix":"64ce108b3e36bd41","entry":"get_DNN_and_trainkwargs","repo":"zlin7/LVD","repo_kind":"official","path":"models/baselines/DNN.py","file_url":"https://github.com/zlin7/LVD/blob/HEAD/models/baselines/DNN.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"64ce108b3e36bd41"}},{"code_sha256_prefix":"f9bc950f0b053082","entry":"get_default_fitkwargs","repo":"zlin7/LVD","repo_kind":"official","path":"demos/experiments.py","file_url":"https://github.com/zlin7/LVD/blob/HEAD/demos/experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9bc950f0b053082"}},{"code_sha256_prefix":"d33ff76452c269e0","entry":"my_split","repo":"zlin7/LVD","repo_kind":"official","path":"demos/experiments.py","file_url":"https://github.com/zlin7/LVD/blob/HEAD/demos/experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d33ff76452c269e0"}},{"code_sha256_prefix":"a08c0e709979c15a","entry":"smoothing","repo":"zlin7/LVD","repo_kind":"official","path":"utils/eval_utils.py","file_url":"https://github.com/zlin7/LVD/blob/HEAD/utils/eval_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a08c0e709979c15a"}},{"code_sha256_prefix":"e56f011f0044c5d0","entry":"summ_df","repo":"zlin7/LVD","repo_kind":"official","path":"utils/eval_utils.py","file_url":"https://github.com/zlin7/LVD/blob/HEAD/utils/eval_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e56f011f0044c5d0"}},{"code_sha256_prefix":"471ab010d45f085c","entry":"weighted_quantile_faster","repo":"zlin7/lvd","repo_kind":"official","path":"models/conformal.py","file_url":"https://github.com/zlin7/lvd/blob/HEAD/models/conformal.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"471ab010d45f085c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}