Papers › MedSTS: A Resource for Clinical Semantic Textual Similarity

MedSTS: A Resource for Clinical Semantic Textual Similarity

28 Aug 2018arXiv:1808.09397archive 2025-07-28

Yanshan Wang, Naveed Afzal, Sunyang Fu, Li-Wei Wang, Feichen Shen, Majid Rastegar-Mojarad, Hongfang Liu

The wide adoption of electronic health records (EHRs) has enabled a wide range of applications leveraging EHR data. However, the meaningful use of EHR data largely depends on our ability to efficiently extract and consolidate information embedded in clinical text where natural language processing (NLP) techniques are essential. Semantic textual similarity (STS) that measures the semantic similarity between text snippets plays a significant role in many NLP applications. In the general NLP domain, STS shared tasks have made available a huge collection of text snippet pairs with manual annotations in various domains. In the clinical domain, STS can enable us to detect and eliminate redundant information that may lead to a reduction in cognitive burden and an improvement in the clinical decision-making process. This paper elaborates our efforts to assemble a resource for STS in the medical domain, MedSTS. It consists of a total of 174,629 sentence pairs gathered from a clinical corpus at Mayo Clinic. A subset of MedSTS (MedSTS_ann) containing 1,068 sentence pairs was annotated by two medical experts with semantic similarity scores of 0-5 (low to high similarity). We further analyzed the medical concepts in the MedSTS corpus, and tested four STS systems on the MedSTS_ann corpus. In the future, we will organize a shared task by releasing the MedSTS_ann corpus to motivate the community to tackle the real world clinical problems.

PaperPDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

ncbi-nlp/BioSentVec mentioned on GitHubNOASSERTIONSyntology: not harvested report
ncbi-nlp/BioWordVec mentioned on GitHubNOASSERTIONSyntology: not harvested report
sai4july/custext mentioned on GitHubpytorchSyntology: not harvested report
xiangyue9607/SanText mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Differential Privacy for Text Analytics via Natural Text Sanitization (arXiv:2106.01221; that paper's own run record: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
yangyucheng000/Paper-4 mindsporeSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning (arXiv:2311.10774; that paper's own run record: 14 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 1 honoured, 2 violated, 6 with no contract checked; 5 where Syntology's instrument failed) · 5 unverified); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Decision MakingSTSSemantic SimilaritySemantic Textual SimilaritySentence

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections