{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multimodal-whole-slide-foundation-model-for","title":"Multimodal Whole Slide Foundation Model for Pathology","arxiv_id":"2411.19666","date":"2024-11-29","proceeding":null,"authors":["Tong Ding","Sophia J. Wagner","Andrew H. Song","Richard J. Chen","Ming Y. Lu","Andrew Zhang","Anurag J. Vaidya","Guillaume Jaume","Muhammad Shaban","Ahrong Kim","Drew F. K. Williamson","Bowen Chen","Cristina Almagro-Perez","Paul Doucet","Sharifa Sahai","Chengkuan Chen","Daisuke Komura","Akihiro Kawabe","Shumpei Ishikawa","Georg Gerber","Tingying Peng","Long Phi Le","Faisal Mahmood"],"abstract":"The field of computational pathology has been transformed with recent advances in foundation models that encode histopathology region-of-interests (ROIs) into versatile and transferable feature representations via self-supervised learning (SSL). However, translating these advancements to address complex clinical challenges at the patient and slide level remains constrained by limited clinical data in disease-specific cohorts, especially for rare clinical conditions. We propose TITAN, a multimodal whole slide foundation model pretrained using 335,645 WSIs via visual self-supervised learning and vision-language alignment with corresponding pathology reports and 423,122 synthetic captions generated from a multimodal generative AI copilot for pathology. Without any finetuning or requiring clinical labels, TITAN can extract general-purpose slide representations and generate pathology reports that generalize to resource-limited clinical scenarios such as rare disease retrieval and cancer prognosis. We evaluate TITAN on diverse clinical tasks and find that TITAN outperforms both ROI and slide foundation models across machine learning settings such as linear probing, few-shot and zero-shot classification, rare cancer retrieval and cross-modal retrieval, and pathology report generation.","url_abs":"https://arxiv.org/abs/2411.19666v1","url_pdf":"https://arxiv.org/pdf/2411.19666v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multimodal-whole-slide-foundation-model-for","repo_url":"https://github.com/mahmoodlab/titan","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"multimodal-whole-slide-foundation-model-for","repo_url":"https://github.com/mahmoodlab/trident","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"cross-modal-retrieval","task_name":"Cross-Modal Retrieval"},{"task_slug":"prognosis","task_name":"Prognosis"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"zero-shot-learning","task_name":"Zero-Shot Learning"},{"task_slug":"model","task_name":"model"},{"task_slug":null,"task_name":"zero-shot-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2411.19666","atlas_url":"https://app.syntology.ai/?focus=2411.19666","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.19666"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mahmoodlab/titan","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mahmoodlab/trident","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"e5ad400fe0abbe9a","entry":"cosine_lr","repo":"mahmoodlab/titan","repo_kind":"official","path":"titan/finetune.py","file_url":"https://github.com/mahmoodlab/titan/blob/HEAD/titan/finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e5ad400fe0abbe9a"}},{"code_sha256_prefix":"ac33556a67aac0c5","entry":"create_mlp","repo":"mahmoodlab/titan","repo_kind":"official","path":"titan/finetune.py","file_url":"https://github.com/mahmoodlab/titan/blob/HEAD/titan/finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ac33556a67aac0c5"}},{"code_sha256_prefix":"35ed8192ced57323","entry":"train","repo":"mahmoodlab/titan","repo_kind":"official","path":"titan/finetune.py","file_url":"https://github.com/mahmoodlab/titan/blob/HEAD/titan/finetune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"35ed8192ced57323"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}