Papers › Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized...

Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized Embeddings

23 Sep 2019arXiv:1909.10430archive 2025-07-28

Gregor Wiedemann, Steffen Remus, Avi Chawla, Chris Biemann

Contextualized word embeddings (CWE) such as provided by ELMo (Peters et al., 2018), Flair NLP (Akbik et al., 2018), or BERT (Devlin et al., 2019) are a major recent innovation in NLP. CWEs provide semantic vector representations of words depending on their respective context. Their advantage over static word embeddings has been shown for a number of tasks, such as text classification, sequence tagging, or machine translation. Since vectors of the same word type can vary depending on the respective context, they implicitly provide a model for word sense disambiguation (WSD). We introduce a simple but effective approach to WSD using a nearest neighbor classification on CWEs. We compare the performance of different CWE models for the task and can report improvements above the current state of the art for two standard WSD benchmark datasets. We further show that the pre-trained BERT model is able to place polysemic words into distinct 'sense' regions of the embedding space, while ELMo and Flair NLP do not seem to possess this ability.

Paper PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

uhh-lt/bert-sense officialmentioned in paperpytorchMIT report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

General Classification Translation Word Sense Disambiguation text-classification

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Word Sense Disambiguation	SemEval 2007 Task 17	kNN-BERT + POS (training corpus: SemCor)	F1	63.17	#7 of 9	Archive leaderboard	report
Word Sense Disambiguation	SemEval 2007 Task 17	kNN-BERT	F1	60.94	#8 of 9	Archive leaderboard	report
Word Sense Disambiguation	SemEval 2007 Task 7	kNN-BERT + POS (training corpus: WNGT)	F1	85.32	#3 of 10	Archive leaderboard	report
Word Sense Disambiguation	SemEval 2007 Task 7	kNN-BERT	F1	81.20	#9 of 10	Archive leaderboard	report
Word Sense Disambiguation	SensEval 2 Lexical Sample	kNN-BERT	F1	76.52	#1 of 3	Archive leaderboard	report
Word Sense Disambiguation	SensEval 3 Lexical Sample	kNN-BERT	F1	80.12	#1 of 4	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Adam Attention Attention Dropout BERT BiLSTM Dense Connections Dropout ELMo LSTM Layer Normalization Linear Layer Linear Warmup With Linear Decay Multi-Head Attention Residual Connection Sigmoid Activation Softmax Tanh Activation Weight Decay WordPiece

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections