Papers › NegBERT: A Transfer Learning Approach for Negation Detection and Scope Resolution

NegBERT: A Transfer Learning Approach for Negation Detection and Scope Resolution

11 Nov 2019LREC 2020 5arXiv:1911.04211archive 2025-07-28

Aditya Khandelwal, Suraj Sawant

Negation is an important characteristic of language, and a major component of information extraction from text. This subtask is of considerable importance to the biomedical domain. Over the years, multiple approaches have been explored to address this problem: Rule-based systems, Machine Learning classifiers, Conditional Random Field Models, CNNs and more recently BiLSTMs. In this paper, we look at applying Transfer Learning to this problem. First, we extensively review previous literature addressing Negation Detection and Scope Resolution across the 3 datasets that have gained popularity over the years: the BioScope Corpus, the Sherlock dataset, and the SFU Review Corpus. We then explore the decision choices involved with using BERT, a popular transfer learning model, for this task, and report state-of-the-art results for scope resolution across all 3 datasets. Our model, referred to as NegBERT, achieves a token level F1 score on scope resolution of 92.36 on the Sherlock dataset, 95.68 on the BioScope Abstracts subcorpus, 91.24 on the BioScope Full Papers subcorpus, 90.95 on the SFU Review Corpus, outperforming the previous state-of-the-art systems by a significant margin. We also analyze the model's generalizability to datasets on which it is not trained.

Paper PDF Conference PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

adityak6798/Transformers-For-Negation-and-Speculation officialmentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Negation Negation Detection Negation Scope Resolution Negation and Speculation Cue Detection Transfer Learning

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Negation Scope Resolution	*sem 2012 Shared Task: Sherlock Dataset	NegBERT	F1	92.36	#2 of 3	Archive leaderboard	report
Negation Scope Resolution	BioScope : Abstracts	NegBERT	F1	95.68	#3 of 3	Archive leaderboard	report
Negation Scope Resolution	BioScope : Full Papers	NegBERT	F1	91.24	#2 of 2	Archive leaderboard	report
Negation Scope Resolution	SFU Review Corpus	NegBERT	F1	90.95	#2 of 2	Archive leaderboard	report
Negation and Speculation Cue Detection	*sem 2012 Shared Task: Sherlock Dataset	NegBERT	F1	92.94	#2 of 2	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Adam Attention Attention Dropout BERT Dense Connections Dropout Layer Normalization Linear Layer Linear Warmup With Linear Decay Multi-Head Attention Residual Connection Softmax Weight Decay WordPiece

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections