MedNLI (Medical Natural Language Inference)

archive 2025-07-28

The MedNLI dataset consists of the sentence pairs developed by Physicians from the Past Medical History section of MIMIC-III clinical notes annotated for Definitely True, Maybe True and Definitely False. The dataset contains 11,232 training, 1,395 development and 1,422 test instances. This provides a natural language inference task (NLI) grounded in the medical history of patients.

Source: MT-Clinical BERT: Scaling Clinical Information Extraction with Multitask Learning Image Source: https://arxiv.org/abs/1904.02181

Homepage Benchmarks Papers JSON

Benchmarks archive 2025-07-28

All 2 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

		First row (archive order)	Paper	Code
Natural Language Inference	MedNLI	ClinicalMosaic Accuracy 86.59	Patient Trajectory Prediction: Integrating Clinical...	MostHumble/PatientTrajectoryForecasting	7	Compare
Few-Shot Learning	MedNLI	CoT-T5-11B (1024 Shot) Accuracy 78.02	The CoT Collection: Improving Zero-shot and Few-shot...	kaistai/cot-collection +1	1	Compare

Papers archive 2025-07-28

8 shown of 8 papers with a leaderboard row on this dataset's benchmarks, newest first. The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset; the archive's count for this dataset is 9. The Syntology column is from Syntology's graph (read 2026-09-24), stated per sample; it is not part of any archive number.

			Date	Samples run Syntology
Patient Trajectory Prediction: Integrating Clinical Notes with Transformers	1	1	25 Feb 2025	not harvested
BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks	1	1	26 May 2023	ran 1 of 8 samples (7 unverified; 7 pointer-only for licence)
The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning	2	1	23 May 2023	not harvested
BioELECTRA:Pretrained Biomedical text Encoder using Discriminators	1	1	11 Jun 2021	not harvested
SciFive: a text-to-text transformer model for biomedical literature	1	1	28 May 2021	ran 0 of 5 samples (5 unverified)
CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characters	2	1	20 Oct 2020	ran 5 of 8 samples (3 unverified)
Saama Research at MEDIQA 2019: Pre-trained BioBERT with Attention Visualisation for Medical Natural Language Inference	0	1	1 Aug 2019	not harvested
Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets	4	1	13 Jun 2019	ran 0 of 2 samples (2 unverified)

Dataset loaders archive 2025-07-28

huggingface.co/datasets/OUTCOMESAI/mednli huggingface/datasetstfpytorchjax

huggingface.co/datasets/bigbio/mednli huggingface/datasetstfpytorchjax

2 loaders as listed in the archive; links are outbound and not re-checked here.

Tasks archive 2025-07-28

Few-Shot Learning Natural Language Inference

License archive 2025-07-28

No licence recorded in the archive. Absence here is not a statement about the dataset's terms.

Modalities archive 2025-07-28

Texts Medical

Languages archive 2025-07-28

No language tagged.

Variants archive 2025-07-28

MedNLI

1 variant name, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections