Methods › Natural Language Processing › Language Models › ELMo
ELMo
Introduced by Matthew E. Peters et al. in Deep contextualized word representations
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Embeddings from Language Models, or ELMo, is a type of deep contextualized word representation that models both (1) complex characteristics of word use (e.g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i.e., to model polysemy). Word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pre-trained on a large text corpus.
A biLM combines both a forward and backward LM. ELMo jointly maximizes the log likelihood of the forward and backward directions. To add ELMo to a supervised model, we freeze the weights of the biLM and then concatenate the ELMo vector ELMOᵗᵃˢᵏₖ with xₖ and pass the ELMO enhanced representation [xₖ; ELMOᵗᵃˢᵏₖ] into the task RNN. Here xₖ is a context-independent token representation for each token position.
Image Source: here
In Syntology about another paper, arXiv:2510.11168: repository known, no verified implementation (declared_code; repositories: 2 listed of 2 in the graph, strongest link: listed only). The lookup matched 'ELMO' ignoring case. About another paper: this answer is about arXiv:2510.11168, while the archive credits arXiv:1802.05365.
Origin paper of this answer, arXiv:2510.11168 (its page on this site · its page on Syntology): Syntology's run record for its harvested code, 8 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 2 honoured, 0 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 2 unverified (of 10 harvested samples). The instrument figure counts failures of Syntology's instrument, not of the code. Ran means executed on a synthesized input, not that the code is correct.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 234 · with a code link: 90 · where Syntology ran a sample: 17 (16 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (17 of 234 tagged: 16 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 3: papers 1 to 100 of 234, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
A Comparative Analysis of Static Word Embeddings for Hungarian 12 May 2025 · 1 repository · arXiv:2505.07809
-
Embedding-based Approaches to Hyperpartisan News Detection 2 Jan 2025 · 0 repositories · arXiv:2501.01370
-
Generative Pretrained Embedding and Hierarchical Irregular Time Series Representation for Daily Living Activity Recognition 27 Dec 2024 · 1 repository · arXiv:2412.19732
-
LLMs are Also Effective Embedding Models: An In-depth Overview 17 Dec 2024 · 0 repositories · arXiv:2412.12591
-
From Word Vectors to Multimodal Embeddings: Techniques, Applications, and Future Directions For Large Language Models 6 Nov 2024 · 0 repositories · arXiv:2411.05036
-
ELMO: Enhanced Real-time LiDAR Motion Capture through Upsampling 9 Oct 2024 · 0 repositories · arXiv:2410.06963
-
Evaluating the Efficacy of AI Techniques in Textual Anonymization: A Comparative Study 9 May 2024 · 0 repositories · arXiv:2405.06709
-
Where exactly does contextualization in a PLM happen? 11 Dec 2023 · 0 repositories · arXiv:2312.06514
-
Semantic Change Detection for the Romanian Language 23 Aug 2023 · 1 repository · arXiv:2308.12131
-
PEvoLM: Protein Sequence Evolutionary Information Language Model 16 Aug 2023 · 1 repository · arXiv:2308.08578
-
On "Scientific Debt" in NLP: A Case for More Rigour in Language Model Pre-Training Research 5 Jun 2023 · 0 repositories · arXiv:2306.02870
-
Analyzing the Generalizability of Deep Contextualized Language Representations For Text Classification 22 Mar 2023 · 0 repositories · arXiv:2303.12936
-
Classifying Text-Based Conspiracy Tweets related to COVID-19 using Contextualized Word Embeddings 7 Mar 2023 · 0 repositories · arXiv:2303.03706
-
CKG: Dynamic Representation Based on Context and Knowledge Graph 9 Dec 2022 · 0 repositories · arXiv:2212.04909
-
A Context-Sensitive Word Embedding Approach for The Detection of Troll Tweets 17 Jul 2022 · 0 repositories · arXiv:2207.08230
-
Always Keep your Target in Mind: Studying Semantics and Improving Performance of Neural Lexical Substitution 7 Jun 2022 · 1 repository · arXiv:2206.11815
-
Parameter-Efficient Tuning by Manipulating Hidden States of Pretrained Language Models For Classification Tasks 10 Apr 2022 · 0 repositories · arXiv:2204.04596
-
Bridging Pre-trained Language Models and Hand-crafted Features for Unsupervised POS Tagging 19 Mar 2022 · 1 repository · arXiv:2203.10315
-
Using Word Embeddings to Analyze Protests News 11 Mar 2022 · 0 repositories · arXiv:2203.05875
-
Assessment of contextualised representations in detecting outcome phrases in clinical trials 13 Feb 2022 · 0 repositories · arXiv:2203.03547
-
"A Passage to India": Pre-trained Word Embeddings for Indian Languages 27 Dec 2021 · 0 repositories · arXiv:2112.13800
-
KARL-Trans-NER: Knowledge Aware Representation Learning for Named Entity Recognition using Transformers 30 Nov 2021 · 0 repositories · arXiv:2111.15436
-
Using Language Model to Bootstrap Human Activity Recognition Ambient Sensors Based in Smart Homes 23 Nov 2021 · 1 repository · arXiv:2111.12158
-
Investigating the Use of BERT Anchors for Bilingual Lexicon Induction with Minimal Supervision 16 Nov 2021 · 0 repositories
-
Fake News Detection in Spanish Using Deep Learning Techniques 13 Oct 2021 · 1 repository · arXiv:2110.06461
-
A Comprehensive Comparison of Word Embeddings in Event & Entity Coreference Resolution 11 Oct 2021 · 1 repository · arXiv:2110.05115
-
Cross-Architecture Distillation Using Bidirectional CMOW Embeddings 29 Sep 2021 · 0 repositories
-
General Cross-Architecture Distillation of Pretrained Language Models into Matrix Embeddings 17 Sep 2021 · 1 repository · arXiv:2109.08449
-
Revisiting Tri-training of Dependency Parsers 16 Sep 2021 · 2 repositories · arXiv:2109.08122
-
Sense representations for Portuguese: experiments with sense embeddings and deep neural language models 31 Aug 2021 · 0 repositories · arXiv:2109.00025
-
Evaluating the Robustness of Neural Language Models to Input Perturbations 27 Aug 2021 · 1 repository · arXiv:2108.12237Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
Curriculum learning for language modeling 4 Aug 2021 · 1 repository · arXiv:2108.02170Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 1 pointer-only (licence)
-
RAW-C: Relatedness of Ambiguous Words in Context (A New Lexical Resource for English) 1 Aug 2021 · 0 repositories
-
ContextNet: A Click-Through Rate Prediction Framework Using Contextual information to Refine Feature Embedding 26 Jul 2021 · 4 repositories · arXiv:2107.12025
-
Evaluation of contextual embeddings on less-resourced languages 22 Jul 2021 · 0 repositories · arXiv:2107.10614
-
Unsupervised Spoken Utterance Classification 2 Jul 2021 · 0 repositories · arXiv:2107.01068
-
Cross-lingual alignments of ELMo contextual embeddings 30 Jun 2021 · 0 repositories · arXiv:2106.15986
-
Clinical Named Entity Recognition using Contextualized Token Representations 23 Jun 2021 · 0 repositories · arXiv:2106.12608
-
RAW-C: Relatedness of Ambiguous Words--in Context (A New Lexical Resource for English) 27 May 2021 · 1 repository · arXiv:2105.13266
-
The Low-Dimensional Linear Geometry of Contextualized Word Representations 15 May 2021 · 0 repositories · arXiv:2105.07109
-
Role of Artificial Intelligence in Detection of Hateful Speech for Hinglish Data on Social Media 11 May 2021 · 0 repositories · arXiv:2105.04913
-
Towards Trustworthy Deception Detection: Benchmarking Model Robustness across Domains, Modalities, and Languages 23 Apr 2021 · 0 repositories · arXiv:2104.11761
-
Large-Scale Contextualised Language Modelling for Norwegian 13 Apr 2021 · 2 repositories · arXiv:2104.06546Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples)
-
Combining Pre-trained Word Embeddings and Linguistic Features for Sequential Metaphor Identification 7 Apr 2021 · 0 repositories · arXiv:2104.03285
-
Evaluating Neural Model Robustness for Machine Comprehension 1 Apr 2021 · 0 repositories
-
Probing for idiomaticity in vector space models 1 Apr 2021 · 1 repository
-
Representing ELMo embeddings as two-dimensional text online 30 Mar 2021 · 0 repositories · arXiv:2103.16414
-
Towards Multi-Sense Cross-Lingual Alignment of Contextual Embeddings 11 Mar 2021 · 1 repository · arXiv:2103.06459
-
CEQE: Contextualized Embeddings for Query Expansion 9 Mar 2021 · 0 repositories · arXiv:2103.05256
-
Few-shot Learning for Slot Tagging with Attentive Relational Network 3 Mar 2021 · 0 repositories · arXiv:2103.02333
-
From Universal Language Model to Downstream Task: Improving RoBERTa-Based Vietnamese Hate Speech Detection 24 Feb 2021 · 0 repositories · arXiv:2102.12162
-
Coloring the Black Box: What Synesthesia Tells Us about Character Embeddings 26 Jan 2021 · 0 repositories · arXiv:2101.10565
-
A Hybrid Approach to Measure Semantic Relatedness in Biomedical Concepts 25 Jan 2021 · 0 repositories · arXiv:2101.10196
-
Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Models 24 Jan 2021 · 1 repository · arXiv:2101.09688
-
Neural Contract Element Extraction Revisited: Letters from Sesame Street 12 Jan 2021 · 0 repositories · arXiv:2101.04355
-
Learning Better Sentence Representation with Syntax Information 9 Jan 2021 · 0 repositories · arXiv:2101.03343
-
Presenting a Dataset for Collaborator Recommending Systems in Academic Social Network: a Case Study on ReseachGate 29 Dec 2020 · 0 repositories · arXiv:2101.01141
-
Improved Biomedical Word Embeddings in the Transformer Era 22 Dec 2020 · 1 repository · arXiv:2012.11808
-
Classifier Probes May Just Learn from Linear Context Features 1 Dec 2020 · 1 repository
-
Cross-lingual Annotation Projection in Legal Texts 1 Dec 2020 · 1 repository
-
DeftPunk at SemEval-2020 Task 6: Using RNN-ensemble for the Sentence Classification. 1 Dec 2020 · 0 repositories
-
Embeddings in Natural Language Processing 1 Dec 2020 · 0 repositories
-
GRUBERT: A GRU-Based Method to Fuse BERT Hidden Layers for Twitter Sentiment Analysis 1 Dec 2020 · 2 repositories
-
LAST at SemEval-2020 Task 10: Finding Tokens to Emphasise in Short Written Texts with Precomputed Embedding Models and LightGBM 1 Dec 2020 · 0 repositories
-
MineriaUNAM at SemEval-2020 Task 3: Predicting Contextual WordSimilarity Using a Centroid Based Approach and Word Embeddings 1 Dec 2020 · 0 repositories
-
NTUAAILS at SemEval-2020 Task 11: Propaganda Detection and Classification with biLSTMs and ELMo 1 Dec 2020 · 0 repositories
-
TUE at SemEval-2020 Task 1: Detecting Semantic Change by Clustering Contextual Word Embeddings 1 Dec 2020 · 0 repositories
-
UniTuebingenCL at SemEval-2020 Task 7: Humor Detection in News Headlines 1 Dec 2020 · 0 repositories
-
Bi-ISCA: Bidirectional Inter-Sentence Contextual Attention Mechanism for Detecting Sarcasm in User Generated Noisy Short Text 23 Nov 2020 · 0 repositories · arXiv:2011.11465
-
Enhance Robustness of Sequence Labelling with Masked Adversarial Training 1 Nov 2020 · 0 repositories
-
The Amazing World of Neural Language Generation 1 Nov 2020 · 0 repositories
-
Seq2Mol: Automatic design of de novo molecules conditioned by the target protein sequences through deep neural networks 29 Oct 2020 · 0 repositories · arXiv:2010.15900
-
Improving Compositional Generalization in Semantic Parsing 12 Oct 2020 · 1 repository · arXiv:2010.05647Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples)
-
Unsupervised Distillation of Syntactic Information from Contextualized Word Representations 11 Oct 2020 · 1 repository · arXiv:2010.05265
-
Masked ELMo: An evolution of ELMo towards fully contextual RNN language models 8 Oct 2020 · 0 repositories · arXiv:2010.04302
-
ELMo and BERT in semantic change detection for Russian 7 Oct 2020 · 0 repositories · arXiv:2010.03481
-
Where Are the Facts? Searching for Fact-checked Information to Alleviate the Spread of Fake News 7 Oct 2020 · 2 repositories · arXiv:2010.03159Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples)
-
Metaphor Detection using Deep Contextualized Word Embeddings 26 Sep 2020 · 0 repositories · arXiv:2009.12565
-
FarsTail: A Persian Natural Language Inference Dataset 18 Sep 2020 · 1 repository · arXiv:2009.08820
-
Multi-modal embeddings using multi-task learning for emotion recognition 10 Sep 2020 · 0 repositories · arXiv:2009.05019
-
Comparative Evaluation of Pretrained Transfer Learning Models on Automatic Short Answer Grading 2 Sep 2020 · 1 repository · arXiv:2009.01303
-
MICE: Mining Idioms with Contextual Embeddings 13 Aug 2020 · 1 repository · arXiv:2008.05759
-
Assessing Demographic Bias in Named Entity Recognition 8 Aug 2020 · 1 repository · arXiv:2008.03415
-
Towards Debiasing Sentence Representations 16 Jul 2020 · 1 repository · arXiv:2007.08100Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
Clinical Concept Linking with Contextualized Neural Representations 1 Jul 2020 · 0 repositories
-
Interpreting Pretrained Contextualized Representations via Reductions to Static Embeddings 1 Jul 2020 · 0 repositories
-
Why is penguin more similar to polar bear than to sea gull? Analyzing conceptual knowledge in distributional models 1 Jul 2020 · 0 repositories
-
Building Interpretable Interaction Trees for Deep NLP Models 29 Jun 2020 · 0 repositories · arXiv:2007.04298
-
A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages 11 Jun 2020 · 0 repositories · arXiv:2006.06202
-
A Comparative Study of Lexical Substitution Approaches based on Neural Language Models 29 May 2020 · 0 repositories · arXiv:2006.00031
-
On Incorporating Structural Information to improve Dialogue Response Generation 28 May 2020 · 1 repository · arXiv:2005.14315
-
MASK: A flexible framework to facilitate de-identification of clinical texts 24 May 2020 · 1 repository · arXiv:2005.11687
-
On the Robustness of Language Encoders against Grammatical Errors 12 May 2020 · 1 repository · arXiv:2005.05683Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluation 9 May 2020 · 0 repositories · arXiv:2005.04322
-
Spying on your neighbors: Fine-grained probing of contextual embeddings for information about surrounding words 4 May 2020 · 0 repositories · arXiv:2005.01810Syntology 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result (of 1 harvested sample) · 1 pointer-only (licence)
-
``A Passage to India'': Pre-trained Word Embeddings for Indian Languages 1 May 2020 · 0 repositories
-
AIA-BDE: A Corpus of FAQs in Portuguese and their Variations 1 May 2020 · 0 repositories
-
Analyzing ELMo and DistilBERT on Socio-political News Classification 1 May 2020 · 0 repositories
-
Facilitating Corpus Usage: Making Icelandic Corpora More Accessible for Researchers and Language Users 1 May 2020 · 0 repositories
-
French Contextualized Word-Embeddings with a sip of CaBeRnet: a New French Balanced Reference Corpus 1 May 2020 · 0 repositories
Tasks archive 2025-07-28
20 shown of 199 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections