{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-graph-transformer-for-whole-slide-image","title":"A graph-transformer for whole slide image classification","arxiv_id":"2205.09671","date":"2022-05-19","proceeding":null,"authors":["Yi Zheng","Rushin H. Gindra","Emily J. Green","Eric J. Burks","Margrit Betke","Jennifer E. Beane","Vijaya B. Kolachalama"],"abstract":"Deep learning is a powerful tool for whole slide image (WSI) analysis. Typically, when performing supervised deep learning, a WSI is divided into small patches, trained and the outcomes are aggregated to estimate disease grade. However, patch-based methods introduce label noise during training by assuming that each patch is independent with the same label as the WSI and neglect overall WSI-level information that is significant in disease grading. Here we present a Graph-Transformer (GT) that fuses a graph-based representation of an WSI and a vision transformer for processing pathology images, called GTP, to predict disease grade. We selected $4,818$ WSIs from the Clinical Proteomic Tumor Analysis Consortium (CPTAC), the National Lung Screening Trial (NLST), and The Cancer Genome Atlas (TCGA), and used GTP to distinguish adenocarcinoma (LUAD) and squamous cell carcinoma (LSCC) from adjacent non-cancerous tissue (normal). First, using NLST data, we developed a contrastive learning framework to generate a feature extractor. This allowed us to compute feature vectors of individual WSI patches, which were used to represent the nodes of the graph followed by construction of the GTP framework. Our model trained on the CPTAC data achieved consistently high performance on three-label classification (normal versus LUAD versus LSCC: mean accuracy$= 91.2$ $\\pm$ $2.5\\%$) based on five-fold cross-validation, and mean accuracy $= 82.3$ $\\pm$ $1.0\\%$ on external test data (TCGA). We also introduced a graph-based saliency mapping technique, called GraphCAM, that can identify regions that are highly associated with the class label. Our findings demonstrate GTP as an interpretable and effective deep learning framework for WSI-level classification.","url_abs":"https://arxiv.org/abs/2205.09671v1","url_pdf":"https://arxiv.org/pdf/2205.09671v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-graph-transformer-for-whole-slide-image","repo_url":"https://github.com/vkola-lab/tmi2022","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2205.09671","atlas_url":"https://app.syntology.ai/?focus=2205.09671","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2205.09671"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vkola-lab/tmi2022","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":8,"ran_fixture":1,"ran_honours":1},"by_repo_kind":{"official":{"samples":10,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"52cdef895690da2c","entry":"adj_matrix","repo":"vkola-lab/tmi2022","repo_kind":"official","path":"feature_extractor/build_graphs.py","file_url":"https://github.com/vkola-lab/tmi2022/blob/HEAD/feature_extractor/build_graphs.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"52cdef895690da2c"}},{"code_sha256_prefix":"e383c38d070af79f","entry":"bag_dataset","repo":"vkola-lab/tmi2022","repo_kind":"official","path":"feature_extractor/build_graphs.py","file_url":"https://github.com/vkola-lab/tmi2022/blob/HEAD/feature_extractor/build_graphs.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e383c38d070af79f"}},{"code_sha256_prefix":"0c4725cf9e1614b0","entry":"cam_to_mask","repo":"vkola-lab/tmi2022","repo_kind":"official","path":"src/vis_graphcam.py","file_url":"https://github.com/vkola-lab/tmi2022/blob/HEAD/src/vis_graphcam.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0c4725cf9e1614b0"}},{"code_sha256_prefix":"cd1f597238e8dea4","entry":"collate","repo":"vkola-lab/tmi2022","repo_kind":"official","path":"helper.py","file_url":"https://github.com/vkola-lab/tmi2022/blob/HEAD/helper.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd1f597238e8dea4"}},{"code_sha256_prefix":"927e684e71ca3ee9","entry":"compute_rollout_attention","repo":"vkola-lab/tmi2022","repo_kind":"official","path":"models/ViT.py","file_url":"https://github.com/vkola-lab/tmi2022/blob/HEAD/models/ViT.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"927e684e71ca3ee9"}},{"code_sha256_prefix":"7d76114278992e2e","entry":"preparefeatureLabel","repo":"vkola-lab/tmi2022","repo_kind":"official","path":"helper.py","file_url":"https://github.com/vkola-lab/tmi2022/blob/HEAD/helper.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7d76114278992e2e"}},{"code_sha256_prefix":"507e144112f81dc6","entry":"safe_divide","repo":"vkola-lab/tmi2022","repo_kind":"official","path":"models/layers.py","file_url":"https://github.com/vkola-lab/tmi2022/blob/HEAD/models/layers.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"507e144112f81dc6"}},{"code_sha256_prefix":"ad7b295017b82497","entry":"show_cam_on_image","repo":"vkola-lab/tmi2022","repo_kind":"official","path":"src/vis_graphcam.py","file_url":"https://github.com/vkola-lab/tmi2022/blob/HEAD/src/vis_graphcam.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ad7b295017b82497"}},{"code_sha256_prefix":"a3cdd23a719c231f","entry":"trunc_normal_","repo":"vkola-lab/tmi2022","repo_kind":"official","path":"models/ViT.py","file_url":"https://github.com/vkola-lab/tmi2022/blob/HEAD/models/ViT.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a3cdd23a719c231f"}},{"code_sha256_prefix":"8ce93d785861725e","entry":"xml_read_labels","repo":"vkola-lab/tmi2022","repo_kind":"official","path":"src/tile_WSI.py","file_url":"https://github.com/vkola-lab/tmi2022/blob/HEAD/src/tile_WSI.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8ce93d785861725e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}