{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2601-15888","title":"Understanding the Transfer Limits of Vision Foundation Models","arxiv_id":"2601.15888","date":"2026-01-22","proceeding":null,"authors":["Shiqi Huang","Yipei Wang","Natasha Thorley","Alexander Ng","Shaheer Saeed","Mark Emberton","Shonit Punwani","Veeru Kasivisvanathan","Dean Barratt","Daniel Alexander","Yipeng Hu"],"abstract":"Foundation models leverage large-scale pretraining to capture extensive knowledge, demonstrating generalization in a wide range of language tasks. By comparison, vision foundation models (VFMs) often exhibit uneven improvements across downstream tasks, despite substantial computational investment. We postulate that this limitation arises from a mismatch between pretraining objectives and the demands of downstream vision-and-imaging tasks. Pretraining strategies like masked image reconstruction or contrastive learning shape representations for tasks such as recovery of generic visual patterns or global semantic structures, which may not align with the task-specific requirements of downstream applications including segmentation, classification, or image synthesis. To investigate this in a concrete real-world clinical area, we assess two VFMs, a reconstruction-focused MAE-based model (ProFound) and a contrastive-learning-based model (ProViCNet), on five prostate multiparametric MR imaging tasks, examining how such task alignment influences transfer performance, i.e., from pretraining to fine-tuning. Our findings indicate that better alignment between pretraining and downstream tasks, measured by simple divergence metrics such as maximum-mean-discrepancy (MMD) between the same features before and after fine-tuning, correlates with greater performance improvements and faster convergence, emphasizing the importance of designing and analyzing pretraining objectives with downstream applicability in mind.","url_abs":"https://arxiv.org/abs/2601.15888","url_pdf":"https://arxiv.org/pdf/2601.15888","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2601.15888","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2601.15888"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/pipiwang/ProFound","reach":{"status":"ok","spdx":"Apache-2.0"}},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/pimed/ProViCNet","reach":null}],"summary":{"ran_honours":1,"ran_draft_wrong":2,"unverified":17},"by_repo_kind":{"found_in_text":{"samples":20,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"91d26d98a8c2bcac","entry":"as_tuple","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/ModelArchitectures/base_vit.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/ModelArchitectures/base_vit.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"91d26d98a8c2bcac"}},{"code_sha256_prefix":"3e9ea4498e4b8a8d","entry":"double_conv","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/ModelArchitectures/unet.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/ModelArchitectures/unet.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e9ea4498e4b8a8d"}},{"code_sha256_prefix":"940661961d8b8781","entry":"drop_head_variant","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/ModelArchitectures/base_vit.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/ModelArchitectures/base_vit.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"940661961d8b8781"}},{"code_sha256_prefix":"a2d409f6ec03d623","entry":"GetModel","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/ModelArchitectures/Models.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/ModelArchitectures/Models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a2d409f6ec03d623"}},{"code_sha256_prefix":"552f28e15ae79272","entry":"OneBatchInference_seg","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/util_functions/train_functions.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/util_functions/train_functions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"552f28e15ae79272"}},{"code_sha256_prefix":"b0c3ffbb35c049a3","entry":"OneBatchTraining_seg","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/util_functions/train_functions.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/util_functions/train_functions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b0c3ffbb35c049a3"}},{"code_sha256_prefix":"85111b4bb4bb5503","entry":"OneBatchTraining_seg_contrastive","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/util_functions/train_functions.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/util_functions/train_functions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"85111b4bb4bb5503"}},{"code_sha256_prefix":"1953a572fdb66e95","entry":"auc_on_batch","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/util_functions/utils_weighted.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/util_functions/utils_weighted.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1953a572fdb66e95"}},{"code_sha256_prefix":"9b572c793221d45e","entry":"b16","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/ModelArchitectures/LeViTUnet/LeViTUnet.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/ModelArchitectures/LeViTUnet/LeViTUnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b572c793221d45e"}},{"code_sha256_prefix":"1a770e29157e6b41","entry":"build_dataset","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/ModelArchitectures/LeViTUnet/datasets.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/ModelArchitectures/LeViTUnet/datasets.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1a770e29157e6b41"}},{"code_sha256_prefix":"3bc74137ab36aa79","entry":"build_transform","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/ModelArchitectures/LeViTUnet/datasets.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/ModelArchitectures/LeViTUnet/datasets.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3bc74137ab36aa79"}},{"code_sha256_prefix":"13557fd29ba64bfd","entry":"check_conditions","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/util_functions/inference.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/util_functions/inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"13557fd29ba64bfd"}},{"code_sha256_prefix":"1296f06da66e11f7","entry":"collate_prostate","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/util_functions/Prostate_DataGenerator.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/util_functions/Prostate_DataGenerator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1296f06da66e11f7"}},{"code_sha256_prefix":"81bca198360c17d7","entry":"collate_prostate_position","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/util_functions/Prostate_DataGenerator.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/util_functions/Prostate_DataGenerator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"81bca198360c17d7"}},{"code_sha256_prefix":"5c61a66c83281894","entry":"collate_prostate_position_CS","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/util_functions/Prostate_DataGenerator.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/util_functions/Prostate_DataGenerator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5c61a66c83281894"}},{"code_sha256_prefix":"129077a6905ca1c9","entry":"dice_coef","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/util_functions/utils_weighted.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/util_functions/utils_weighted.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"129077a6905ca1c9"}},{"code_sha256_prefix":"cb16d40a894f6fe4","entry":"iou_on_batch","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/util_functions/utils_weighted.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/util_functions/utils_weighted.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb16d40a894f6fe4"}},{"code_sha256_prefix":"0167a2f2bbae916f","entry":"keep_csPCa_only","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/util_functions/inference.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/util_functions/inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0167a2f2bbae916f"}},{"code_sha256_prefix":"8c053b1e8b60dcc6","entry":"load_pretrained_weights","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/ModelArchitectures/base_vit.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/ModelArchitectures/base_vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8c053b1e8b60dcc6"}},{"code_sha256_prefix":"c10ef1f374c0ff39","entry":"model_factory_v3_NM","repo":"pimed/ProViCNet","repo_kind":"found_in_text","path":"ProViCNet/ModelArchitectures/LeViTUnet/LeViTUnet.py","file_url":"https://github.com/pimed/ProViCNet/blob/HEAD/ProViCNet/ModelArchitectures/LeViTUnet/LeViTUnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c10ef1f374c0ff39"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CV","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}