{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2603-23794","title":"Sparse Autoencoders for Interpretable Medical Image Representation Learning","arxiv_id":"2603.23794","date":"2026-03-24","proceeding":null,"authors":["Philipp Wesp","Robbie Holland","Vasiliki Sideri-Lampretsa","Sergios Gatidis"],"abstract":"Vision foundation models (FMs) achieve state-of-the-art performance in medical imaging. However, they encode information in abstract latent representations that clinicians cannot interrogate or verify. The goal of this study is to investigate Sparse Autoencoders (SAEs) for replacing opaque FM image representations with human-interpretable, sparse features. We train SAEs on embeddings from BiomedParse (biomedical) and DINOv3 (general-purpose) using 909,873 CT and MRI 2D image slices from the TotalSegmentator dataset. We find that learned sparse features: (a) reconstruct original embeddings with high fidelity (R2 up to 0.941) and recover up to 87.8% of downstream performance using only 10 features (99.4% dimensionality reduction), (b) preserve semantic fidelity in image retrieval tasks, (c) correspond to specific concepts that can be expressed in language using large language model (LLM)-based auto-interpretation. (d) bridge clinical language and abstract latent representations in zero-shot language-driven image retrieval. Our work indicates SAEs are a promising pathway towards interpretable, concept-driven medical vision systems. Code repository: https://github.com/pwesp/sail.","url_abs":"https://arxiv.org/abs/2603.23794","url_pdf":"https://arxiv.org/pdf/2603.23794","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2603.23794","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2603.23794"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/pwesp/sail","reach":null}],"summary":{"unverified":11},"by_repo_kind":{"found_in_text":{"samples":11,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"98d7b05264eec734","entry":"clean_concept_description","repo":"pwesp/sail","repo_kind":"found_in_text","path":"src/autointerp.py","file_url":"https://github.com/pwesp/sail/blob/HEAD/src/autointerp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"98d7b05264eec734"}},{"code_sha256_prefix":"d3bca2d9014db5cd","entry":"extract_top_k_dense_features","repo":"pwesp/sail","repo_kind":"found_in_text","path":"src/fingerprint_retrieval.py","file_url":"https://github.com/pwesp/sail/blob/HEAD/src/fingerprint_retrieval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d3bca2d9014db5cd"}},{"code_sha256_prefix":"ff869560566344a4","entry":"extract_top_k_sparse_features","repo":"pwesp/sail","repo_kind":"found_in_text","path":"src/fingerprint_retrieval.py","file_url":"https://github.com/pwesp/sail/blob/HEAD/src/fingerprint_retrieval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ff869560566344a4"}},{"code_sha256_prefix":"dc0cee3539cfdc09","entry":"find_always_active_features","repo":"pwesp/sail","repo_kind":"found_in_text","path":"src/fingerprint_retrieval.py","file_url":"https://github.com/pwesp/sail/blob/HEAD/src/fingerprint_retrieval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dc0cee3539cfdc09"}},{"code_sha256_prefix":"444f91992acc54c4","entry":"format_time","repo":"pwesp/sail","repo_kind":"found_in_text","path":"src/downstream_clf.py","file_url":"https://github.com/pwesp/sail/blob/HEAD/src/downstream_clf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"444f91992acc54c4"}},{"code_sha256_prefix":"4ef4b1e30ee8bb9c","entry":"get_image_path","repo":"pwesp/sail","repo_kind":"found_in_text","path":"src/dataloading.py","file_url":"https://github.com/pwesp/sail/blob/HEAD/src/dataloading.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4ef4b1e30ee8bb9c"}},{"code_sha256_prefix":"eba0805256b1bfc2","entry":"get_reference_metadata","repo":"pwesp/sail","repo_kind":"found_in_text","path":"src/dataloading.py","file_url":"https://github.com/pwesp/sail/blob/HEAD/src/dataloading.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"eba0805256b1bfc2"}},{"code_sha256_prefix":"b05325d59532790c","entry":"load_vlm_model","repo":"pwesp/sail","repo_kind":"found_in_text","path":"src/autointerp.py","file_url":"https://github.com/pwesp/sail/blob/HEAD/src/autointerp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b05325d59532790c"}},{"code_sha256_prefix":"1489116d7926a86c","entry":"map_organs_to_anatomical_regions","repo":"pwesp/sail","repo_kind":"found_in_text","path":"src/clinical_region_mapping.py","file_url":"https://github.com/pwesp/sail/blob/HEAD/src/clinical_region_mapping.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1489116d7926a86c"}},{"code_sha256_prefix":"d2c0e7176210ae61","entry":"setup_job_logging","repo":"pwesp/sail","repo_kind":"found_in_text","path":"src/downstream_clf.py","file_url":"https://github.com/pwesp/sail/blob/HEAD/src/downstream_clf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d2c0e7176210ae61"}},{"code_sha256_prefix":"14794569c3c48c42","entry":"setup_target_group_logging","repo":"pwesp/sail","repo_kind":"found_in_text","path":"src/downstream_clf.py","file_url":"https://github.com/pwesp/sail/blob/HEAD/src/downstream_clf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"14794569c3c48c42"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}