{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/calibrated-predictive-distributions-via","title":"Towards Instance-Wise Calibration: Local Amortized Diagnostics and Reshaping of Conditional Densities (LADaR)","arxiv_id":"2205.14568","date":"2022-05-29","proceeding":null,"authors":["Biprateep Dey","David Zhao","Brett H. Andrews","Jeffrey A. Newman","Rafael Izbicki","Ann B. Lee"],"abstract":"Key science questions, such as galaxy distance and weather forecasting, often require knowing the full predictive distribution of a target variable $y$ given complex inputs $\\mathbf{x}$. Despite recent advances in machine learning and physics-based models, it remains challenging to assess whether an initial model is calibrated for all $\\mathbf{x}$, and when needed, to reshape the densities of $y$ toward \"instance-wise\" calibration. This paper introduces the LADaR (Local Amortized Diagnostics and Reshaping of Conditional Densities) framework and proposes a new computationally efficient algorithm ($\\texttt{Cal-PIT}$) that produces interpretable local diagnostics and provides a mechanism for adjusting conditional density estimates (CDEs). $\\texttt{Cal-PIT}$ learns a single interpretable local probability--probability (optimal transport) map from calibration data that identifies where and how the initial model is miscalibrated across feature space, which can be used to morph CDEs such that they are well-calibrated. We illustrate the LADaR framework on synthetic examples, including probabilistic forecasting from image sequences, akin to predicting storm wind speed from satellite imagery. Our main science application involves estimating the probability density functions of galaxy distances given photometric data, where $\\texttt{Cal-PIT}$ achieves better instance-wise calibration than all 11 other literature methods in a benchmark data challenge, demonstrating its utility for next-generation cosmological analyses.","url_abs":"https://arxiv.org/abs/2205.14568v7","url_pdf":"https://arxiv.org/pdf/2205.14568v7.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"calibrated-predictive-distributions-via","repo_url":"https://github.com/lee-group-cmu/cal-pit","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"conformal-prediction","task_name":"Conformal Prediction"},{"task_slug":"density-estimation","task_name":"Density Estimation"},{"task_slug":"morph","task_name":"MORPH"},{"task_slug":"photometric-redshift-estimation","task_name":"Photometric Redshift Estimation"},{"task_slug":"uncertainty-quantification","task_name":"Uncertainty Quantification"},{"task_slug":"weather-forecasting","task_name":"Weather Forecasting"},{"task_slug":"regression-1","task_name":"regression"}],"methods":[{"method_slug":"normalizing-flows","method_name":"Normalizing Flows"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2205.14568","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2205.14568"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lee-group-cmu/cal-pit","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":13},"by_repo_kind":{"official":{"samples":13,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7f2c17e50d5be9d9","entry":"cde_loss","repo":"lee-group-cmu/cal-pit","repo_kind":"official","path":"src/calpit/metrics.py","file_url":"https://github.com/lee-group-cmu/cal-pit/blob/HEAD/src/calpit/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f2c17e50d5be9d9"}},{"code_sha256_prefix":"83efca9add7d68a9","entry":"cde_loss","repo":"lee-group-cmu/cal-pit","repo_kind":"official","path":"src/calpit/nn/utils.py","file_url":"https://github.com/lee-group-cmu/cal-pit/blob/HEAD/src/calpit/nn/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"83efca9add7d68a9"}},{"code_sha256_prefix":"0760afa3939fa275","entry":"clipped_relu","repo":"lee-group-cmu/cal-pit","repo_kind":"official","path":"src/calpit/nn/umnn/MonotonicNN.py","file_url":"https://github.com/lee-group-cmu/cal-pit/blob/HEAD/src/calpit/nn/umnn/MonotonicNN.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0760afa3939fa275"}},{"code_sha256_prefix":"8097965387fc479c","entry":"computeIntegrand","repo":"lee-group-cmu/cal-pit","repo_kind":"official","path":"src/calpit/nn/umnn/NeuralIntegral.py","file_url":"https://github.com/lee-group-cmu/cal-pit/blob/HEAD/src/calpit/nn/umnn/NeuralIntegral.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8097965387fc479c"}},{"code_sha256_prefix":"7e8243c235d7660a","entry":"computeIntegrand","repo":"lee-group-cmu/cal-pit","repo_kind":"official","path":"src/calpit/nn/umnn/ParallelNeuralIntegral.py","file_url":"https://github.com/lee-group-cmu/cal-pit/blob/HEAD/src/calpit/nn/umnn/ParallelNeuralIntegral.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7e8243c235d7660a"}},{"code_sha256_prefix":"dc4044ceb6fafbbd","entry":"compute_cc_weights","repo":"lee-group-cmu/cal-pit","repo_kind":"official","path":"src/calpit/nn/umnn/NeuralIntegral.py","file_url":"https://github.com/lee-group-cmu/cal-pit/blob/HEAD/src/calpit/nn/umnn/NeuralIntegral.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dc4044ceb6fafbbd"}},{"code_sha256_prefix":"308155c2926cf9a5","entry":"cramer_von_mises_statistic","repo":"lee-group-cmu/cal-pit","repo_kind":"official","path":"src/calpit/metrics.py","file_url":"https://github.com/lee-group-cmu/cal-pit/blob/HEAD/src/calpit/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"308155c2926cf9a5"}},{"code_sha256_prefix":"f10bf9b7644b9a05","entry":"integrate","repo":"lee-group-cmu/cal-pit","repo_kind":"official","path":"src/calpit/nn/umnn/NeuralIntegral.py","file_url":"https://github.com/lee-group-cmu/cal-pit/blob/HEAD/src/calpit/nn/umnn/NeuralIntegral.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f10bf9b7644b9a05"}},{"code_sha256_prefix":"84a59e5fddc7a99f","entry":"integrate","repo":"lee-group-cmu/cal-pit","repo_kind":"official","path":"src/calpit/nn/umnn/ParallelNeuralIntegral.py","file_url":"https://github.com/lee-group-cmu/cal-pit/blob/HEAD/src/calpit/nn/umnn/ParallelNeuralIntegral.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"84a59e5fddc7a99f"}},{"code_sha256_prefix":"afcc65f183fe3db8","entry":"kolmogorov_smirnov_statistic","repo":"lee-group-cmu/cal-pit","repo_kind":"official","path":"src/calpit/metrics.py","file_url":"https://github.com/lee-group-cmu/cal-pit/blob/HEAD/src/calpit/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"afcc65f183fe3db8"}},{"code_sha256_prefix":"d7e1d9a95d046a6c","entry":"normalize","repo":"lee-group-cmu/cal-pit","repo_kind":"official","path":"src/calpit/utils.py","file_url":"https://github.com/lee-group-cmu/cal-pit/blob/HEAD/src/calpit/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d7e1d9a95d046a6c"}},{"code_sha256_prefix":"8a8f5e827db8d81b","entry":"trapz_grid","repo":"lee-group-cmu/cal-pit","repo_kind":"official","path":"src/calpit/utils.py","file_url":"https://github.com/lee-group-cmu/cal-pit/blob/HEAD/src/calpit/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8a8f5e827db8d81b"}},{"code_sha256_prefix":"7c8062702c2083ca","entry":"trapz_grid_torch","repo":"lee-group-cmu/cal-pit","repo_kind":"official","path":"src/calpit/utils.py","file_url":"https://github.com/lee-group-cmu/cal-pit/blob/HEAD/src/calpit/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7c8062702c2083ca"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}