{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-uncertainty-estimation-in-active-learning","title":"On uncertainty estimation in active learning for image segmentation","arxiv_id":"2007.06364","date":"2020-07-13","proceeding":null,"authors":["Bo Li","Tommy Sonne Alstrøm"],"abstract":"Uncertainty estimation is important for interpreting the trustworthiness of machine learning models in many applications. This is especially critical in the data-driven active learning setting where the goal is to achieve a certain accuracy with minimum labeling effort. In such settings, the model learns to select the most informative unlabeled samples for annotation based on its estimated uncertainty. The highly uncertain predictions are assumed to be more informative for improving model performance. In this paper, we explore uncertainty calibration within an active learning framework for medical image segmentation, an area where labels often are scarce. Various uncertainty estimation methods and acquisition strategies (regions and full images) are investigated. We observe that selecting regions to annotate instead of full images leads to more well-calibrated models. Additionally, we experimentally show that annotating regions can cut 50% of pixels that need to be labeled by humans compared to annotating full images.","url_abs":"https://arxiv.org/abs/2007.06364v1","url_pdf":"https://arxiv.org/pdf/2007.06364v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-uncertainty-estimation-in-active-learning","repo_url":"https://github.com/lyn1874/region_based_active_learning","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"active-learning","task_name":"Active Learning"},{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"medical-image-segmentation","task_name":"Medical Image Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2007.06364","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2007.06364"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lyn1874/region_based_active_learning","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_violates":1,"unverified":17},"by_repo_kind":{"official":{"samples":18,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7c508037b40522af","entry":"str2bool","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"visualize_calibration_score.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/visualize_calibration_score.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7c508037b40522af"}},{"code_sha256_prefix":"50fc13a68b64379f","entry":"ax_global_get","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"visualize_calibration_score.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/visualize_calibration_score.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"50fc13a68b64379f"}},{"code_sha256_prefix":"d2b70422b98f5661","entry":"calc_auc_score","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"optimization/loss_region_specific.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/optimization/loss_region_specific.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d2b70422b98f5661"}},{"code_sha256_prefix":"78f559a5e9a51a8b","entry":"calc_f1_score","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"optimization/loss_region_specific.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/optimization/loss_region_specific.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"78f559a5e9a51a8b"}},{"code_sha256_prefix":"029462f1307151a0","entry":"calc_uncertainty","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"eval_calibration/calc_calibration_score.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/eval_calibration/calc_calibration_score.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"029462f1307151a0"}},{"code_sha256_prefix":"a767dc35a3f40491","entry":"calculate_score_for_patch","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"models/acquisition_region.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/models/acquisition_region.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a767dc35a3f40491"}},{"code_sha256_prefix":"a3a2aaecc255402f","entry":"conv_layer_renew","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"models/layer_utils.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/models/layer_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a3a2aaecc255402f"}},{"code_sha256_prefix":"12d68c73441b00a9","entry":"deconv_layer_renew","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"models/layer_utils.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/models/layer_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"12d68c73441b00a9"}},{"code_sha256_prefix":"e140b3c7b79e9645","entry":"extract_bald_index","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"models/acquistion_full_image.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/models/acquistion_full_image.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e140b3c7b79e9645"}},{"code_sha256_prefix":"87aacc015d138bc6","entry":"extract_entropy_index","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"models/acquistion_full_image.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/models/acquistion_full_image.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"87aacc015d138bc6"}},{"code_sha256_prefix":"8c7ef5678477ae92","entry":"extract_uncertainty_index","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"models/acquistion_full_image.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/models/acquistion_full_image.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8c7ef5678477ae92"}},{"code_sha256_prefix":"1229928c1c5f8499","entry":"get_deconv_layer_weight","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"models/layer_utils.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/models/layer_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1229928c1c5f8499"}},{"code_sha256_prefix":"1ea8d82f1e501df8","entry":"get_time","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"eval_calibration/calc_calibration_score.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/eval_calibration/calc_calibration_score.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1ea8d82f1e501df8"}},{"code_sha256_prefix":"486f99c1fc0d5a72","entry":"give_score_path","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"visualize_calibration_score.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/visualize_calibration_score.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"486f99c1fc0d5a72"}},{"code_sha256_prefix":"05677c6af8f1ea2d","entry":"return_pseudo_label","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"models/acquisition_region.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/models/acquisition_region.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"05677c6af8f1ea2d"}},{"code_sha256_prefix":"c1fe50bec09abed1","entry":"select_most_uncertain_patch","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"models/acquisition_region.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/models/acquisition_region.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c1fe50bec09abed1"}},{"code_sha256_prefix":"95bd0e68182d1059","entry":"tf_f1score","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"optimization/loss_region_specific.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/optimization/loss_region_specific.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"95bd0e68182d1059"}},{"code_sha256_prefix":"16bcb3740ed61a09","entry":"transfer_numeric_index_back_to_imindex","repo":"lyn1874/region_based_active_learning","repo_kind":"official","path":"eval_calibration/calc_calibration_score.py","file_url":"https://github.com/lyn1874/region_based_active_learning/blob/HEAD/eval_calibration/calc_calibration_score.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"16bcb3740ed61a09"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}