{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/training-like-a-medical-resident-universal","title":"Training Like a Medical Resident: Context-Prior Learning Toward Universal Medical Image Segmentation","arxiv_id":"2306.02416","date":"2023-06-04","proceeding":"CVPR 2024 1","authors":["Yunhe Gao","Zhuowei Li","Di Liu","Mu Zhou","Shaoting Zhang","Dimitris N. Metaxas"],"abstract":"A major focus of clinical imaging workflow is disease diagnosis and management, leading to medical imaging datasets strongly tied to specific clinical objectives. This scenario has led to the prevailing practice of developing task-specific segmentation models, without gaining insights from widespread imaging cohorts. Inspired by the training program of medical radiology residents, we propose a shift towards universal medical image segmentation, a paradigm aiming to build medical image understanding foundation models by leveraging the diversity and commonality across clinical targets, body regions, and imaging modalities. Towards this goal, we develop Hermes, a novel context-prior learning approach to address the challenges of data heterogeneity and annotation differences in medical image segmentation. In a large collection of eleven diverse datasets (2,438 3D images) across five modalities (CT, PET, T1, T2 and cine MRI) and multiple body regions, we demonstrate the merit of the universal paradigm over the traditional paradigm on addressing multiple tasks within a single model. By exploiting the synergy across tasks, Hermes achieves state-of-the-art performance on all testing datasets and shows superior model scalability. Results on two additional datasets reveals Hermes' strong performance for transfer learning, incremental learning, and generalization to downstream tasks. Hermes's learned priors demonstrate an appealing trait to reflect the intricate relations among tasks and modalities, which aligns with the established anatomical and imaging principles in radiology. The code is available: https://github.com/yhygao/universal-medical-image-segmentation.","url_abs":"https://arxiv.org/abs/2306.02416v3","url_pdf":"https://arxiv.org/pdf/2306.02416v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"training-like-a-medical-resident-universal","repo_url":"https://github.com/yhygao/universal-medical-image-segmentation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"training-like-a-medical-resident-universal","repo_url":"https://github.com/yhygao/UTNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"incremental-learning","task_name":"Incremental Learning"},{"task_slug":"medical-image-segmentation","task_name":"Medical Image Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2306.02416","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.02416"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yhygao/UTNet","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yhygao/universal-medical-image-segmentation","reach":{"status":"ok"}}],"summary":{"ran_fixture":1,"ran_draft_wrong":3,"unverified":7},"by_repo_kind":{"listed":{"samples":11,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7d8eaf5537f699e4","entry":"np2th","repo":"yhygao/UTNet","repo_kind":"listed","path":"model/transunet.py","file_url":"https://github.com/yhygao/UTNet/blob/HEAD/model/transunet.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":2,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7d8eaf5537f699e4"}},{"code_sha256_prefix":"0f786c407fb1ee4c","entry":"swish","repo":"yhygao/UTNet","repo_kind":"listed","path":"model/transunet.py","file_url":"https://github.com/yhygao/UTNet/blob/HEAD/model/transunet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f786c407fb1ee4c"}},{"code_sha256_prefix":"be3fea8e6f5db9c7","entry":"conv1x1","repo":"yhygao/UTNet","repo_kind":"listed","path":"model/conv_trans_utils.py","file_url":"https://github.com/yhygao/UTNet/blob/HEAD/model/conv_trans_utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"be3fea8e6f5db9c7"}},{"code_sha256_prefix":"583f9780bdd00a45","entry":"conv3x3","repo":"yhygao/UTNet","repo_kind":"listed","path":"model/conv_trans_utils.py","file_url":"https://github.com/yhygao/UTNet/blob/HEAD/model/conv_trans_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"583f9780bdd00a45"}},{"code_sha256_prefix":"f538eb32f0f1cb58","entry":"compute_average_surface_distance","repo":"yhygao/UTNet","repo_kind":"listed","path":"utils/metrics.py","file_url":"https://github.com/yhygao/UTNet/blob/HEAD/utils/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f538eb32f0f1cb58"}},{"code_sha256_prefix":"130f8faa5f58f861","entry":"compute_robust_hausdorff","repo":"yhygao/UTNet","repo_kind":"listed","path":"utils/metrics.py","file_url":"https://github.com/yhygao/UTNet/blob/HEAD/utils/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"130f8faa5f58f861"}},{"code_sha256_prefix":"129f12535b5d7f43","entry":"compute_surface_distances","repo":"yhygao/UTNet","repo_kind":"listed","path":"utils/metrics.py","file_url":"https://github.com/yhygao/UTNet/blob/HEAD/utils/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"129f12535b5d7f43"}},{"code_sha256_prefix":"df61ad5db47c8fdb","entry":"create_table_neighbour_code_to_contour_length","repo":"yhygao/UTNet","repo_kind":"listed","path":"utils/lookup_tables.py","file_url":"https://github.com/yhygao/UTNet/blob/HEAD/utils/lookup_tables.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df61ad5db47c8fdb"}},{"code_sha256_prefix":"bb47471cf1d0e670","entry":"create_table_neighbour_code_to_surface_area","repo":"yhygao/UTNet","repo_kind":"listed","path":"utils/lookup_tables.py","file_url":"https://github.com/yhygao/UTNet/blob/HEAD/utils/lookup_tables.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bb47471cf1d0e670"}},{"code_sha256_prefix":"d82a40639bf451e5","entry":"window_partition","repo":"yhygao/UTNet","repo_kind":"listed","path":"model/swin_unet.py","file_url":"https://github.com/yhygao/UTNet/blob/HEAD/model/swin_unet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d82a40639bf451e5"}},{"code_sha256_prefix":"503511a5414aca33","entry":"window_reverse","repo":"yhygao/UTNet","repo_kind":"listed","path":"model/swin_unet.py","file_url":"https://github.com/yhygao/UTNet/blob/HEAD/model/swin_unet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"503511a5414aca33"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}