{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/robust-and-efficient-medical-imaging-with","title":"Robust and Efficient Medical Imaging with Self-Supervision","arxiv_id":"2205.09723","date":"2022-05-19","proceeding":null,"authors":["Shekoofeh Azizi","Laura Culp","Jan Freyberg","Basil Mustafa","Sebastien Baur","Simon Kornblith","Ting Chen","Patricia MacWilliams","S. Sara Mahdavi","Ellery Wulczyn","Boris Babenko","Megan Wilson","Aaron Loh","Po-Hsuan Cameron Chen","YuAn Liu","Pinal Bavishi","Scott Mayer McKinney","Jim Winkens","Abhijit Guha Roy","Zach Beaver","Fiona Ryan","Justin Krogue","Mozziyar Etemadi","Umesh Telang","Yun Liu","Lily Peng","Greg S. Corrado","Dale R. Webster","David Fleet","Geoffrey Hinton","Neil Houlsby","Alan Karthikesalingam","Mohammad Norouzi","Vivek Natarajan"],"abstract":"Recent progress in Medical Artificial Intelligence (AI) has delivered systems that can reach clinical expert level performance. However, such systems tend to demonstrate sub-optimal \"out-of-distribution\" performance when evaluated in clinical settings different from the training environment. A common mitigation strategy is to develop separate systems for each clinical setting using site-specific data [1]. However, this quickly becomes impractical as medical data is time-consuming to acquire and expensive to annotate [2]. Thus, the problem of \"data-efficient generalization\" presents an ongoing difficulty for Medical AI development. Although progress in representation learning shows promise, their benefits have not been rigorously studied, specifically for out-of-distribution settings. To meet these challenges, we present REMEDIS, a unified representation learning strategy to improve robustness and data-efficiency of medical imaging AI. REMEDIS uses a generic combination of large-scale supervised transfer learning with self-supervised learning and requires little task-specific customization. We study a diverse range of medical imaging tasks and simulate three realistic application scenarios using retrospective data. REMEDIS exhibits significantly improved in-distribution performance with up to 11.5% relative improvement in diagnostic accuracy over a strong supervised baseline. More importantly, our strategy leads to strong data-efficient generalization of medical imaging AI, matching strong supervised baselines using between 1% to 33% of retraining data across tasks. These results suggest that REMEDIS can significantly accelerate the life-cycle of medical imaging AI development thereby presenting an important step forward for medical imaging AI to deliver broad impact.","url_abs":"https://arxiv.org/abs/2205.09723v2","url_pdf":"https://arxiv.org/pdf/2205.09723v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"robust-and-efficient-medical-imaging-with","repo_url":"https://github.com/google-research/big_transfer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"jax","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"robust-and-efficient-medical-imaging-with","repo_url":"https://github.com/google-research/medical-ai-research-foundations","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"robust-and-efficient-medical-imaging-with","repo_url":"https://github.com/mahmoodlab/hest","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"diagnostic","task_name":"Diagnostic"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2205.09723","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2205.09723"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mahmoodlab/hest","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-research/medical-ai-research-foundations","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-research/big_transfer","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":4,"ran_violates":1,"unverified":23},"by_repo_kind":{"official":{"samples":16,"ran":3,"repositories":1},"listed":{"samples":12,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8924ee1d1cde05c4","entry":"add_contrastive_loss","repo":"google-research/medical-ai-research-foundations","repo_kind":"listed","path":"objective.py","file_url":"https://github.com/google-research/medical-ai-research-foundations/blob/HEAD/objective.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8924ee1d1cde05c4"}},{"code_sha256_prefix":"ee80cf467065076b","entry":"conv1x1","repo":"google-research/big_transfer","repo_kind":"official","path":"bit_pytorch/models.py","file_url":"https://github.com/google-research/big_transfer/blob/HEAD/bit_pytorch/models.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ee80cf467065076b"}},{"code_sha256_prefix":"f01b8d3901f0b289","entry":"conv3x3","repo":"google-research/big_transfer","repo_kind":"official","path":"bit_pytorch/models.py","file_url":"https://github.com/google-research/big_transfer/blob/HEAD/bit_pytorch/models.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f01b8d3901f0b289"}},{"code_sha256_prefix":"7023eb86801bb3a3","entry":"tf2th","repo":"google-research/big_transfer","repo_kind":"official","path":"bit_pytorch/models.py","file_url":"https://github.com/google-research/big_transfer/blob/HEAD/bit_pytorch/models.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7023eb86801bb3a3"}},{"code_sha256_prefix":"7aac0de5d1568953","entry":"tpu_cross_replica_concat","repo":"google-research/medical-ai-research-foundations","repo_kind":"listed","path":"objective.py","file_url":"https://github.com/google-research/medical-ai-research-foundations/blob/HEAD/objective.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7aac0de5d1568953"}},{"code_sha256_prefix":"113df01dadeca915","entry":"add_name_prefix","repo":"google-research/big_transfer","repo_kind":"official","path":"bit_tf2/models.py","file_url":"https://github.com/google-research/big_transfer/blob/HEAD/bit_tf2/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"113df01dadeca915"}},{"code_sha256_prefix":"8af989f02c354e14","entry":"add_supervised_loss","repo":"google-research/medical-ai-research-foundations","repo_kind":"listed","path":"objective.py","file_url":"https://github.com/google-research/medical-ai-research-foundations/blob/HEAD/objective.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8af989f02c354e14"}},{"code_sha256_prefix":"d38eeb648f68f197","entry":"create_dat","repo":"google-research/big_transfer","repo_kind":"official","path":"bit_pytorch/lbtoolbox.py","file_url":"https://github.com/google-research/big_transfer/blob/HEAD/bit_pytorch/lbtoolbox.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d38eeb648f68f197"}},{"code_sha256_prefix":"a068248c1b6c4117","entry":"dropblock","repo":"google-research/medical-ai-research-foundations","repo_kind":"listed","path":"resnet.py","file_url":"https://github.com/google-research/medical-ai-research-foundations/blob/HEAD/resnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a068248c1b6c4117"}},{"code_sha256_prefix":"6cc1fe250eeb4811","entry":"equalize_histogram_fn","repo":"google-research/medical-ai-research-foundations","repo_kind":"listed","path":"data_util.py","file_url":"https://github.com/google-research/medical-ai-research-foundations/blob/HEAD/data_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6cc1fe250eeb4811"}},{"code_sha256_prefix":"eec365515fbb2119","entry":"find_fewshot_indices","repo":"google-research/big_transfer","repo_kind":"official","path":"bit_pytorch/fewshot.py","file_url":"https://github.com/google-research/big_transfer/blob/HEAD/bit_pytorch/fewshot.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"eec365515fbb2119"}},{"code_sha256_prefix":"55be7cb60b3a523c","entry":"find_indices_loader","repo":"google-research/big_transfer","repo_kind":"official","path":"bit_pytorch/fewshot.py","file_url":"https://github.com/google-research/big_transfer/blob/HEAD/bit_pytorch/fewshot.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"55be7cb60b3a523c"}},{"code_sha256_prefix":"d65ebdb640d62fc2","entry":"fixed_padding","repo":"google-research/big_transfer","repo_kind":"official","path":"bit_jax/models.py","file_url":"https://github.com/google-research/big_transfer/blob/HEAD/bit_jax/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d65ebdb640d62fc2"}},{"code_sha256_prefix":"b1b73febd494c9b7","entry":"fixed_padding","repo":"google-research/medical-ai-research-foundations","repo_kind":"listed","path":"resnet.py","file_url":"https://github.com/google-research/medical-ai-research-foundations/blob/HEAD/resnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b1b73febd494c9b7"}},{"code_sha256_prefix":"7d63034d0bfadd78","entry":"get_mixup","repo":"google-research/big_transfer","repo_kind":"official","path":"bit_hyperrule.py","file_url":"https://github.com/google-research/big_transfer/blob/HEAD/bit_hyperrule.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7d63034d0bfadd78"}},{"code_sha256_prefix":"a933bb17fdb8ebb4","entry":"get_resolution","repo":"google-research/big_transfer","repo_kind":"official","path":"bit_hyperrule.py","file_url":"https://github.com/google-research/big_transfer/blob/HEAD/bit_hyperrule.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a933bb17fdb8ebb4"}},{"code_sha256_prefix":"53256b67d70b0daf","entry":"get_resolution_from_dataset","repo":"google-research/big_transfer","repo_kind":"official","path":"bit_hyperrule.py","file_url":"https://github.com/google-research/big_transfer/blob/HEAD/bit_hyperrule.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"53256b67d70b0daf"}},{"code_sha256_prefix":"14ec8ff095316493","entry":"get_train_steps","repo":"google-research/medical-ai-research-foundations","repo_kind":"listed","path":"model_util.py","file_url":"https://github.com/google-research/medical-ai-research-foundations/blob/HEAD/model_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"14ec8ff095316493"}},{"code_sha256_prefix":"65a7c9919024b795","entry":"group_normalize","repo":"google-research/big_transfer","repo_kind":"official","path":"bit_tf2/normalization.py","file_url":"https://github.com/google-research/big_transfer/blob/HEAD/bit_tf2/normalization.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"65a7c9919024b795"}},{"code_sha256_prefix":"fb45bd9f63767265","entry":"learning_rate_schedule","repo":"google-research/medical-ai-research-foundations","repo_kind":"listed","path":"model_util.py","file_url":"https://github.com/google-research/medical-ai-research-foundations/blob/HEAD/model_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fb45bd9f63767265"}},{"code_sha256_prefix":"c2be9d262aacedc9","entry":"linear_layer","repo":"google-research/medical-ai-research-foundations","repo_kind":"listed","path":"model_util.py","file_url":"https://github.com/google-research/medical-ai-research-foundations/blob/HEAD/model_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c2be9d262aacedc9"}},{"code_sha256_prefix":"12109eac56e48b6b","entry":"load_dat","repo":"google-research/big_transfer","repo_kind":"official","path":"bit_pytorch/lbtoolbox.py","file_url":"https://github.com/google-research/big_transfer/blob/HEAD/bit_pytorch/lbtoolbox.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"12109eac56e48b6b"}},{"code_sha256_prefix":"0c26c80b6286f5ec","entry":"pad_to_batch","repo":"google-research/medical-ai-research-foundations","repo_kind":"listed","path":"data.py","file_url":"https://github.com/google-research/medical-ai-research-foundations/blob/HEAD/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0c26c80b6286f5ec"}},{"code_sha256_prefix":"d04f187058cb2a9c","entry":"random_apply","repo":"google-research/medical-ai-research-foundations","repo_kind":"listed","path":"data_util.py","file_url":"https://github.com/google-research/medical-ai-research-foundations/blob/HEAD/data_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d04f187058cb2a9c"}},{"code_sha256_prefix":"6c4f36fab52ddbe9","entry":"random_brightness","repo":"google-research/medical-ai-research-foundations","repo_kind":"listed","path":"data_util.py","file_url":"https://github.com/google-research/medical-ai-research-foundations/blob/HEAD/data_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6c4f36fab52ddbe9"}},{"code_sha256_prefix":"1614afcca20ebedf","entry":"sample_subset","repo":"google-research/big_transfer","repo_kind":"official","path":"input_pipeline_tf2_or_jax.py","file_url":"https://github.com/google-research/big_transfer/blob/HEAD/input_pipeline_tf2_or_jax.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1614afcca20ebedf"}},{"code_sha256_prefix":"12323f0640436acc","entry":"setup_logger","repo":"google-research/big_transfer","repo_kind":"official","path":"bit_common.py","file_url":"https://github.com/google-research/big_transfer/blob/HEAD/bit_common.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"12323f0640436acc"}},{"code_sha256_prefix":"c272f024d815d59d","entry":"standardize","repo":"google-research/big_transfer","repo_kind":"official","path":"bit_jax/models.py","file_url":"https://github.com/google-research/big_transfer/blob/HEAD/bit_jax/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c272f024d815d59d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}