{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/calibration-with-bias-corrected-temperature","title":"Maximum Likelihood with Bias-Corrected Calibration is Hard-To-Beat at Label Shift Adaptation","arxiv_id":"1901.06852","date":"2019-01-21","proceeding":null,"authors":["Amr Alexandari","Anshul Kundaje","Avanti Shrikumar"],"abstract":"Label shift refers to the phenomenon where the prior class probability p(y) changes between the training and test distributions, while the conditional probability p(x|y) stays fixed. Label shift arises in settings like medical diagnosis, where a classifier trained to predict disease given symptoms must be adapted to scenarios where the baseline prevalence of the disease is different. Given estimates of p(y|x) from a predictive model, Saerens et al. proposed an efficient maximum likelihood algorithm to correct for label shift that does not require model retraining, but a limiting assumption of this algorithm is that p(y|x) is calibrated, which is not true of modern neural networks. Recently, Black Box Shift Learning (BBSL) and Regularized Learning under Label Shifts (RLLS) have emerged as state-of-the-art techniques to cope with label shift when a classifier does not output calibrated probabilities, but both methods require model retraining with importance weights and neither has been benchmarked against maximum likelihood. Here we (1) show that combining maximum likelihood with a type of calibration we call bias-corrected calibration outperforms both BBSL and RLLS across diverse datasets and distribution shifts, (2) prove that the maximum likelihood objective is concave, and (3) introduce a principled strategy for estimating source-domain priors that improves robustness to poor calibration. This work demonstrates that the maximum likelihood with appropriate calibration is a formidable and efficient baseline for label shift adaptation; notebooks reproducing experiments available at https://github.com/kundajelab/labelshiftexperiments","url_abs":"https://arxiv.org/abs/1901.06852v5","url_pdf":"https://arxiv.org/pdf/1901.06852v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"calibration-with-bias-corrected-temperature","repo_url":"https://github.com/kundajelab/abstention","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"calibration-with-bias-corrected-temperature","repo_url":"https://github.com/blindauth/labelshiftexperiments","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"calibration-with-bias-corrected-temperature","repo_url":"https://github.com/kundajelab/labelshiftexperiments","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"diabetic-retinopathy-detection","task_name":"Diabetic Retinopathy Detection"},{"task_slug":"domain-adaptation","task_name":"Domain Adaptation"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"medical-diagnosis","task_name":"Medical Diagnosis"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1901.06852","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1901.06852"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/blindauth/labelshiftexperiments","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kundajelab/labelshiftexperiments","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kundajelab/abstention","reach":null}],"summary":{"ran_honours":3,"ran_violates":2,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":6,"ran":6,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"5e59c9676de44ba2","entry":"inverse_softmax","repo":"kundajelab/abstention","repo_kind":"official","path":"abstention/calibration.py","file_url":"https://github.com/kundajelab/abstention/blob/HEAD/abstention/calibration.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5e59c9676de44ba2"}},{"code_sha256_prefix":"f0841f73e3d32ff7","entry":"map_to_softmax_format_if_appropriate","repo":"kundajelab/abstention","repo_kind":"official","path":"abstention/calibration.py","file_url":"https://github.com/kundajelab/abstention/blob/HEAD/abstention/calibration.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f0841f73e3d32ff7"}},{"code_sha256_prefix":"0f722b60f8c59c77","entry":"read_labels","repo":"kundajelab/labelshiftexperiments","repo_kind":"official","path":"labelshiftexperiments/cifarandmnist.py","file_url":"https://github.com/kundajelab/labelshiftexperiments/blob/HEAD/labelshiftexperiments/cifarandmnist.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0f722b60f8c59c77"}},{"code_sha256_prefix":"61f3ee33a7b2e137","entry":"read_preds","repo":"kundajelab/labelshiftexperiments","repo_kind":"official","path":"labelshiftexperiments/cifarandmnist.py","file_url":"https://github.com/kundajelab/labelshiftexperiments/blob/HEAD/labelshiftexperiments/cifarandmnist.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"61f3ee33a7b2e137"}},{"code_sha256_prefix":"d069a5bc888c8966","entry":"sample_from_probs_arr","repo":"kundajelab/labelshiftexperiments","repo_kind":"official","path":"labelshiftexperiments/cifarandmnist.py","file_url":"https://github.com/kundajelab/labelshiftexperiments/blob/HEAD/labelshiftexperiments/cifarandmnist.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d069a5bc888c8966"}},{"code_sha256_prefix":"1a0b58e25479df38","entry":"softmax","repo":"kundajelab/abstention","repo_kind":"official","path":"abstention/calibration.py","file_url":"https://github.com/kundajelab/abstention/blob/HEAD/abstention/calibration.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1a0b58e25479df38"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}