{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-14654","title":"Beyond Instance-Level Self-Supervision in 3D Multi-Modal Medical Imaging","arxiv_id":"2605.14654","date":"2026-05-14","proceeding":"ICML","authors":["Tan Pan","Shuhao Mei","Yixuan Sun","Kaiyu Guo","Chen Jiang","Zhaorui Tan","Mengzhu Li","Limei Han","Xiang Zou","Yuan Cheng","Mahsa Baktashmotlagh"],"abstract":"Self-supervised pre-training methods in medical imaging typically treat each individual as an isolated instance, learning representations through augmentation-based objectives or masked reconstruction. They often do not adequately capitalize on a key characteristic of physiological features: anatomical structures maintain consistent spatial relationships across individuals (instances), such as the thalamus being medial to the basal ganglia, regardless of variations in brain size, shape, or pathology. We propose leveraging this cross-instance topological consistency as a supervisory signal. The challenge arises from the inherent variability in medical imaging, which can differ significantly across instances and modalities. To tackle this, we focus on two alignment regimes. (i) Intra-instance: with pixel-level correspondences available, a cross-modal triplet objective explicitly preserves local neighborhood topology. (ii) Inter-instance: without such supervision, we derive pseudo-correspondences to control partial neighborhood alignment and prevent topology collapse across modalities. We validate our approach across 7 downstream multi-modal tasks, achieving average improvements of 1.1% and 5.94% in segmentation and classification tasks, respectively, and demonstrating significantly better robustness when modalities are missing at test time.","url_abs":"https://arxiv.org/abs/2605.14654","url_pdf":"https://arxiv.org/pdf/2605.14654","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2605.14654","atlas_url":"https://app.syntology.ai/?focus=2605.14654","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.14654"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/Ashespt/TACO","reach":null}],"summary":{"ran":11,"ran_draft_wrong":3,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":16,"ran":14,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":16,"samples":[{"code_sha256_prefix":"7ffc55a8aced3b32","entry":"MLP","repo":"Ashespt/TACO","repo_kind":"found_in_text","path":"pretrain/TACO/losses/superglue.py","file_url":"https://github.com/Ashespt/TACO/blob/HEAD/pretrain/TACO/losses/superglue.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7ffc55a8aced3b32"}},{"code_sha256_prefix":"127ac20ae2475310","entry":"apply_cached_rotary_emb","repo":"Ashespt/TACO","repo_kind":"found_in_text","path":"pretrain/TACO/models/modules.py","file_url":"https://github.com/Ashespt/TACO/blob/HEAD/pretrain/TACO/models/modules.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"127ac20ae2475310"}},{"code_sha256_prefix":"a3f7220c6bc5e56c","entry":"attention","repo":"Ashespt/TACO","repo_kind":"found_in_text","path":"pretrain/TACO/losses/superglue.py","file_url":"https://github.com/Ashespt/TACO/blob/HEAD/pretrain/TACO/losses/superglue.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a3f7220c6bc5e56c"}},{"code_sha256_prefix":"c8d2c320d0a1e021","entry":"cos_dist","repo":"Ashespt/TACO","repo_kind":"found_in_text","path":"pretrain/TACO/models/taco.py","file_url":"https://github.com/Ashespt/TACO/blob/HEAD/pretrain/TACO/models/taco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c8d2c320d0a1e021"}},{"code_sha256_prefix":"1a0cccdae6b03369","entry":"exp_map_euclid_to_poincare","repo":"Ashespt/TACO","repo_kind":"found_in_text","path":"pretrain/TACO/models/taco_uni.py","file_url":"https://github.com/Ashespt/TACO/blob/HEAD/pretrain/TACO/models/taco_uni.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1a0cccdae6b03369"}},{"code_sha256_prefix":"76fe02ea3eab4889","entry":"hyp_dist","repo":"Ashespt/TACO","repo_kind":"found_in_text","path":"pretrain/TACO/models/taco_uni.py","file_url":"https://github.com/Ashespt/TACO/blob/HEAD/pretrain/TACO/models/taco_uni.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"76fe02ea3eab4889"}},{"code_sha256_prefix":"cc33ad7787b81d90","entry":"label_to_membership","repo":"Ashespt/TACO","repo_kind":"found_in_text","path":"pretrain/TACO/losses/coderating.py","file_url":"https://github.com/Ashespt/TACO/blob/HEAD/pretrain/TACO/losses/coderating.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cc33ad7787b81d90"}},{"code_sha256_prefix":"46818e125ab57432","entry":"normalize_keypoints","repo":"Ashespt/TACO","repo_kind":"found_in_text","path":"pretrain/TACO/losses/superglue.py","file_url":"https://github.com/Ashespt/TACO/blob/HEAD/pretrain/TACO/losses/superglue.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"46818e125ab57432"}},{"code_sha256_prefix":"8e0ae67329911da7","entry":"normalize_keypoints","repo":"Ashespt/TACO","repo_kind":"found_in_text","path":"pretrain/TACO/models/modules.py","file_url":"https://github.com/Ashespt/TACO/blob/HEAD/pretrain/TACO/models/modules.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8e0ae67329911da7"}},{"code_sha256_prefix":"f0709ca52def958c","entry":"one_hot","repo":"Ashespt/TACO","repo_kind":"found_in_text","path":"pretrain/TACO/losses/coderating.py","file_url":"https://github.com/Ashespt/TACO/blob/HEAD/pretrain/TACO/losses/coderating.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f0709ca52def958c"}},{"code_sha256_prefix":"943180610ec1e5f8","entry":"pairwise_dist","repo":"Ashespt/TACO","repo_kind":"found_in_text","path":"pretrain/TACO/models/taco.py","file_url":"https://github.com/Ashespt/TACO/blob/HEAD/pretrain/TACO/models/taco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"943180610ec1e5f8"}},{"code_sha256_prefix":"30bd5856d6813ffb","entry":"proj","repo":"Ashespt/TACO","repo_kind":"found_in_text","path":"pretrain/TACO/models/taco_uni.py","file_url":"https://github.com/Ashespt/TACO/blob/HEAD/pretrain/TACO/models/taco_uni.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"30bd5856d6813ffb"}},{"code_sha256_prefix":"304a32b8a1f0e8c0","entry":"random_masking","repo":"Ashespt/TACO","repo_kind":"found_in_text","path":"pretrain/TACO/models/decoder.py","file_url":"https://github.com/Ashespt/TACO/blob/HEAD/pretrain/TACO/models/decoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"304a32b8a1f0e8c0"}},{"code_sha256_prefix":"a3ec43c12656acbb","entry":"rotate_half","repo":"Ashespt/TACO","repo_kind":"found_in_text","path":"pretrain/TACO/models/modules.py","file_url":"https://github.com/Ashespt/TACO/blob/HEAD/pretrain/TACO/models/modules.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a3ec43c12656acbb"}},{"code_sha256_prefix":"2bd8efecb1106371","entry":"neighborhood_ranking_crossbrain","repo":"Ashespt/TACO","repo_kind":"found_in_text","path":"pretrain/TACO/models/taco.py","file_url":"https://github.com/Ashespt/TACO/blob/HEAD/pretrain/TACO/models/taco.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2bd8efecb1106371"}},{"code_sha256_prefix":"55d30bd5f483ce56","entry":"uniformer_base","repo":"Ashespt/TACO","repo_kind":"found_in_text","path":"pretrain/TACO/models/uniformer_blocks.py","file_url":"https://github.com/Ashespt/TACO/blob/HEAD/pretrain/TACO/models/uniformer_blocks.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"55d30bd5f483ce56"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CV","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}