Methods › Natural Language Processing › Language Models › ELMo
ELMo
Introduced by Matthew E. Peters et al. in Deep contextualized word representations
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Embeddings from Language Models, or ELMo, is a type of deep contextualized word representation that models both (1) complex characteristics of word use (e.g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i.e., to model polysemy). Word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pre-trained on a large text corpus.
A biLM combines both a forward and backward LM. ELMo jointly maximizes the log likelihood of the forward and backward directions. To add ELMo to a supervised model, we freeze the weights of the biLM and then concatenate the ELMo vector ELMOᵗᵃˢᵏₖ with xₖ and pass the ELMO enhanced representation [xₖ; ELMOᵗᵃˢᵏₖ] into the task RNN. Here xₖ is a context-independent token representation for each token position.
Image Source: here
Papers archive 2025-07-28
30 shown of 234, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
A Comparative Analysis of Static Word Embeddings for Hungarian 12 May 2025 · 1 repository · arXiv:2505.07809
-
Embedding-based Approaches to Hyperpartisan News Detection 2 Jan 2025 · 0 repositories · arXiv:2501.01370
-
Generative Pretrained Embedding and Hierarchical Irregular Time Series Representation for Daily Living Activity Recognition 27 Dec 2024 · 1 repository · arXiv:2412.19732
-
LLMs are Also Effective Embedding Models: An In-depth Overview 17 Dec 2024 · 0 repositories · arXiv:2412.12591
-
From Word Vectors to Multimodal Embeddings: Techniques, Applications, and Future Directions For Large Language Models 6 Nov 2024 · 0 repositories · arXiv:2411.05036
-
ELMO: Enhanced Real-time LiDAR Motion Capture through Upsampling 9 Oct 2024 · 0 repositories · arXiv:2410.06963
-
Evaluating the Efficacy of AI Techniques in Textual Anonymization: A Comparative Study 9 May 2024 · 0 repositories · arXiv:2405.06709
-
Where exactly does contextualization in a PLM happen? 11 Dec 2023 · 0 repositories · arXiv:2312.06514
-
Semantic Change Detection for the Romanian Language 23 Aug 2023 · 1 repository · arXiv:2308.12131
-
PEvoLM: Protein Sequence Evolutionary Information Language Model 16 Aug 2023 · 1 repository · arXiv:2308.08578
-
On "Scientific Debt" in NLP: A Case for More Rigour in Language Model Pre-Training Research 5 Jun 2023 · 0 repositories · arXiv:2306.02870
-
Analyzing the Generalizability of Deep Contextualized Language Representations For Text Classification 22 Mar 2023 · 0 repositories · arXiv:2303.12936
-
Classifying Text-Based Conspiracy Tweets related to COVID-19 using Contextualized Word Embeddings 7 Mar 2023 · 0 repositories · arXiv:2303.03706
-
CKG: Dynamic Representation Based on Context and Knowledge Graph 9 Dec 2022 · 0 repositories · arXiv:2212.04909
-
A Context-Sensitive Word Embedding Approach for The Detection of Troll Tweets 17 Jul 2022 · 0 repositories · arXiv:2207.08230
-
Always Keep your Target in Mind: Studying Semantics and Improving Performance of Neural Lexical Substitution 7 Jun 2022 · 1 repository · arXiv:2206.11815
-
Parameter-Efficient Tuning by Manipulating Hidden States of Pretrained Language Models For Classification Tasks 10 Apr 2022 · 0 repositories · arXiv:2204.04596
-
Bridging Pre-trained Language Models and Hand-crafted Features for Unsupervised POS Tagging 19 Mar 2022 · 1 repository · arXiv:2203.10315
-
Using Word Embeddings to Analyze Protests News 11 Mar 2022 · 0 repositories · arXiv:2203.05875
-
Assessment of contextualised representations in detecting outcome phrases in clinical trials 13 Feb 2022 · 0 repositories · arXiv:2203.03547
-
"A Passage to India": Pre-trained Word Embeddings for Indian Languages 27 Dec 2021 · 0 repositories · arXiv:2112.13800
-
KARL-Trans-NER: Knowledge Aware Representation Learning for Named Entity Recognition using Transformers 30 Nov 2021 · 0 repositories · arXiv:2111.15436
-
Using Language Model to Bootstrap Human Activity Recognition Ambient Sensors Based in Smart Homes 23 Nov 2021 · 1 repository · arXiv:2111.12158
-
Investigating the Use of BERT Anchors for Bilingual Lexicon Induction with Minimal Supervision 16 Nov 2021 · 0 repositories
-
Fake News Detection in Spanish Using Deep Learning Techniques 13 Oct 2021 · 1 repository · arXiv:2110.06461
-
A Comprehensive Comparison of Word Embeddings in Event & Entity Coreference Resolution 11 Oct 2021 · 1 repository · arXiv:2110.05115
-
Cross-Architecture Distillation Using Bidirectional CMOW Embeddings 29 Sep 2021 · 0 repositories
-
General Cross-Architecture Distillation of Pretrained Language Models into Matrix Embeddings 17 Sep 2021 · 1 repository · arXiv:2109.08449
-
Revisiting Tri-training of Dependency Parsers 16 Sep 2021 · 2 repositories · arXiv:2109.08122
-
Sense representations for Portuguese: experiments with sense embeddings and deep neural language models 31 Aug 2021 · 0 repositories · arXiv:2109.00025
Tasks archive 2025-07-28
20 shown of 199 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections