{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/do-not-trust-what-you-trust-miscalibration-in","title":"Do not trust what you trust: Miscalibration in Semi-supervised Learning","arxiv_id":"2403.15567","date":"2024-03-22","proceeding":null,"authors":["Shambhavi Mishra","Balamurali Murugesan","Ismail Ben Ayed","Marco Pedersoli","Jose Dolz"],"abstract":"State-of-the-art semi-supervised learning (SSL) approaches rely on highly confident predictions to serve as pseudo-labels that guide the training on unlabeled samples. An inherent drawback of this strategy stems from the quality of the uncertainty estimates, as pseudo-labels are filtered only based on their degree of uncertainty, regardless of the correctness of their predictions. Thus, assessing and enhancing the uncertainty of network predictions is of paramount importance in the pseudo-labeling process. In this work, we empirically demonstrate that SSL methods based on pseudo-labels are significantly miscalibrated, and formally demonstrate the minimization of the min-entropy, a lower bound of the Shannon entropy, as a potential cause for miscalibration. To alleviate this issue, we integrate a simple penalty term, which enforces the logit distances of the predictions on unlabeled samples to remain low, preventing the network predictions to become overconfident. Comprehensive experiments on a variety of SSL image classification benchmarks demonstrate that the proposed solution systematically improves the calibration performance of relevant SSL models, while also enhancing their discriminative power, being an appealing addition to tackle SSL tasks.","url_abs":"https://arxiv.org/abs/2403.15567v1","url_pdf":"https://arxiv.org/pdf/2403.15567v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"do-not-trust-what-you-trust-miscalibration-in","repo_url":"https://github.com/shambhavicodes/miscalibration-ssl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2403.15567","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.15567"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shambhavicodes/miscalibration-ssl","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":7,"ran_honours":1,"ran_draft_wrong":4,"unverified":1},"by_repo_kind":{"official":{"samples":13,"ran":12,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"dc61eede543c56fc","entry":"array_to_bytes","repo":"shambhavicodes/miscalibration-ssl","repo_kind":"official","path":"preprocess/preprocess_superb.py","file_url":"https://github.com/shambhavicodes/miscalibration-ssl/blob/HEAD/preprocess/preprocess_superb.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dc61eede543c56fc"}},{"code_sha256_prefix":"518ef932a7896b3d","entry":"average_calibration_error","repo":"shambhavicodes/miscalibration-ssl","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/shambhavicodes/miscalibration-ssl/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"518ef932a7896b3d"}},{"code_sha256_prefix":"b19e0120c50d4fd0","entry":"bytes_to_array","repo":"shambhavicodes/miscalibration-ssl","repo_kind":"official","path":"preprocess/preprocess_superb.py","file_url":"https://github.com/shambhavicodes/miscalibration-ssl/blob/HEAD/preprocess/preprocess_superb.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b19e0120c50d4fd0"}},{"code_sha256_prefix":"3346c66c873fff6a","entry":"comatch_contrastive_loss","repo":"shambhavicodes/miscalibration-ssl","repo_kind":"official","path":"semilearn/algorithms/comatch/comatch.py","file_url":"https://github.com/shambhavicodes/miscalibration-ssl/blob/HEAD/semilearn/algorithms/comatch/comatch.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3346c66c873fff6a"}},{"code_sha256_prefix":"2249c8712edfa8ce","entry":"expected_calibration_error","repo":"shambhavicodes/miscalibration-ssl","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/shambhavicodes/miscalibration-ssl/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2249c8712edfa8ce"}},{"code_sha256_prefix":"887e7bb16dc8c6b5","entry":"maximum_calibration_error","repo":"shambhavicodes/miscalibration-ssl","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/shambhavicodes/miscalibration-ssl/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"887e7bb16dc8c6b5"}},{"code_sha256_prefix":"88f4962784e09b88","entry":"param_groups_weight_decay","repo":"shambhavicodes/miscalibration-ssl","repo_kind":"official","path":"semilearn/nets/utils.py","file_url":"https://github.com/shambhavicodes/miscalibration-ssl/blob/HEAD/semilearn/nets/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"88f4962784e09b88"}},{"code_sha256_prefix":"d954c448e1eee8c4","entry":"read_TextClassificationDatasets","repo":"shambhavicodes/miscalibration-ssl","repo_kind":"official","path":"preprocess/preprocess_aclimdb.py","file_url":"https://github.com/shambhavicodes/miscalibration-ssl/blob/HEAD/preprocess/preprocess_aclimdb.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d954c448e1eee8c4"}},{"code_sha256_prefix":"eed03e5ff5639e09","entry":"read_aclImdb","repo":"shambhavicodes/miscalibration-ssl","repo_kind":"official","path":"preprocess/preprocess_aclimdb.py","file_url":"https://github.com/shambhavicodes/miscalibration-ssl/blob/HEAD/preprocess/preprocess_aclimdb.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eed03e5ff5639e09"}},{"code_sha256_prefix":"4c0668013863ad3f","entry":"resize_pos_embed_vit","repo":"shambhavicodes/miscalibration-ssl","repo_kind":"official","path":"semilearn/nets/utils.py","file_url":"https://github.com/shambhavicodes/miscalibration-ssl/blob/HEAD/semilearn/nets/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4c0668013863ad3f"}},{"code_sha256_prefix":"4331f5778c460265","entry":"rotate_img","repo":"shambhavicodes/miscalibration-ssl","repo_kind":"official","path":"semilearn/algorithms/crmatch/crmatch.py","file_url":"https://github.com/shambhavicodes/miscalibration-ssl/blob/HEAD/semilearn/algorithms/crmatch/crmatch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4331f5778c460265"}},{"code_sha256_prefix":"05dda2f18675eb59","entry":"select_data","repo":"shambhavicodes/miscalibration-ssl","repo_kind":"official","path":"preprocess/preprocess_aclimdb.py","file_url":"https://github.com/shambhavicodes/miscalibration-ssl/blob/HEAD/preprocess/preprocess_aclimdb.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"05dda2f18675eb59"}},{"code_sha256_prefix":"b158ea331635a6c9","entry":"load_checkpoint","repo":"shambhavicodes/miscalibration-ssl","repo_kind":"official","path":"semilearn/nets/utils.py","file_url":"https://github.com/shambhavicodes/miscalibration-ssl/blob/HEAD/semilearn/nets/utils.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b158ea331635a6c9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}