{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generalizable-whole-slide-image","title":"Generalizable Whole Slide Image Classification with Fine-Grained Visual-Semantic Interaction","arxiv_id":"2402.19326","date":"2024-02-29","proceeding":"CVPR 2024 1","authors":["Hao Li","Ying Chen","Yifei Chen","Wenxian Yang","Bowen Ding","Yuchen Han","Liansheng Wang","Rongshan Yu"],"abstract":"Whole Slide Image (WSI) classification is often formulated as a Multiple Instance Learning (MIL) problem. Recently, Vision-Language Models (VLMs) have demonstrated remarkable performance in WSI classification. However, existing methods leverage coarse-grained pathogenetic descriptions for visual representation supervision, which are insufficient to capture the complex visual appearance of pathogenetic images, hindering the generalizability of models on diverse downstream tasks. Additionally, processing high-resolution WSIs can be computationally expensive. In this paper, we propose a novel \"Fine-grained Visual-Semantic Interaction\" (FiVE) framework for WSI classification. It is designed to enhance the model's generalizability by leveraging the interaction between localized visual patterns and fine-grained pathological semantics. Specifically, with meticulously designed queries, we start by utilizing a large language model to extract fine-grained pathological descriptions from various non-standardized raw reports. The output descriptions are then reconstructed into fine-grained labels used for training. By introducing a Task-specific Fine-grained Semantics (TFS) module, we enable prompts to capture crucial visual information in WSIs, which enhances representation learning and augments generalization capabilities significantly. Furthermore, given that pathological visual patterns are redundantly distributed across tissue slices, we sample a subset of visual instances during training. Our method demonstrates robust generalizability and strong transferability, dominantly outperforming the counterparts on the TCGA Lung Cancer dataset with at least 9.19% higher accuracy in few-shot experiments. The code is available at: https://github.com/ls1rius/WSI_FiVE.","url_abs":"https://arxiv.org/abs/2402.19326v2","url_pdf":"https://arxiv.org/pdf/2402.19326v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generalizable-whole-slide-image","repo_url":"https://github.com/ls1rius/wsi_five","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"large-language-model","task_name":"Large Language Model"},{"task_slug":"multiple-instance-learning","task_name":"Multiple Instance Learning"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.19326","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.19326"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ls1rius/wsi_five","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/ls1rius/WSI","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}}],"summary":{"ran":5,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":7,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"4573b507cb31c116","entry":"MedCLIPTextModel","repo":"ls1rius/wsi_five","repo_kind":"official","path":"models/FiVE.py","file_url":"https://github.com/ls1rius/wsi_five/blob/HEAD/models/FiVE.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4573b507cb31c116"}},{"code_sha256_prefix":"89e6bbd05a45bb7b","entry":"MultiPatchIntegrationTransformer","repo":"ls1rius/wsi_five","repo_kind":"official","path":"models/FiVE.py","file_url":"https://github.com/ls1rius/wsi_five/blob/HEAD/models/FiVE.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"89e6bbd05a45bb7b"}},{"code_sha256_prefix":"2c2731c44c1cd7d6","entry":"PatchFusionTransformer","repo":"ls1rius/wsi_five","repo_kind":"official","path":"models/FiVE.py","file_url":"https://github.com/ls1rius/wsi_five/blob/HEAD/models/FiVE.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2c2731c44c1cd7d6"}},{"code_sha256_prefix":"fe38f5b5e9b6ed8c","entry":"VisionTransformer","repo":"ls1rius/wsi_five","repo_kind":"official","path":"models/FiVE.py","file_url":"https://github.com/ls1rius/wsi_five/blob/HEAD/models/FiVE.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fe38f5b5e9b6ed8c"}},{"code_sha256_prefix":"f0c7e5aa4d58f2ca","entry":"constants","repo":"ls1rius/wsi_five","repo_kind":"official","path":"models/FiVE.py","file_url":"https://github.com/ls1rius/wsi_five/blob/HEAD/models/FiVE.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f0c7e5aa4d58f2ca"}},{"code_sha256_prefix":"80bec81aa2a30535","entry":"get_pos_embed","repo":"ls1rius/wsi_five","repo_kind":"official","path":"models/FiVE.py","file_url":"https://github.com/ls1rius/wsi_five/blob/HEAD/models/FiVE.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"80bec81aa2a30535"}},{"code_sha256_prefix":"a5796184be02d5ea","entry":"FiVE","repo":"ls1rius/wsi_five","repo_kind":"official","path":"models/FiVE.py","file_url":"https://github.com/ls1rius/wsi_five/blob/HEAD/models/FiVE.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a5796184be02d5ea"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}