Methods › General › Regularization › Embedding Dropout
Embedding Dropout
Introduced by Yarin Gal et al. in A Theoretically Grounded Application of Dropout in Recurrent Neural Networks
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Embedding Dropout is equivalent to performing dropout on the embedding matrix at a word level, where the dropout is broadcast across all the word vector’s embedding. The remaining non-dropped-out word embeddings are scaled by 1/(1-pₑ) where pₑ is the probability of embedding dropout. As the dropout occurs on the embedding matrix that is used for a full forward and backward pass, this means that all occurrences of a specific word will disappear within that pass, equivalent to performing variational dropout on the connection between the one-hot embedding and the embedding lookup.
Source: Merity et al, Regularizing and Optimizing LSTM Language Models
In Syntology name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.
Near matches the lookup lists, each a different name: Positional Embedding Dropout (PED); Rep-Dropout; H-embedding; PR-Embedding; KV-Embedding; Loop Dropout; late dropout; DROPOUT-GRPO.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 64 · with a code link: 32 · where Syntology ran a sample: 6 (5 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (6 of 64 tagged: 5 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 64 of 64, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
Advanced Deep Learning Techniques for Analyzing Earnings Call Transcripts: Methodologies and Applications 27 Feb 2025 · 0 repositories · arXiv:2503.01886
-
No Argument Left Behind: Overlapping Chunks for Faster Processing of Arbitrarily Long Legal Texts 24 Oct 2024 · 0 repositories · arXiv:2410.19184
-
Leveraging Audio-Only Data for Text-Queried Target Sound Extraction 20 Sep 2024 · 0 repositories · arXiv:2409.13152
-
Robust Decision Transformer: Tackling Data Corruption in Offline RL via Sequence Modeling 5 Jul 2024 · 0 repositories · arXiv:2407.04285
-
RICo: Reddit ideological communities 5 Jun 2024 · 1 repository
-
Exploring Multi-Level Threats in Telegram Data with AI-Human Annotation: A Preliminary Study 15 Dec 2023 · 0 repositories
-
Illicit Darkweb Classification via Natural-language Processing: Classifying Illicit Content of Webpages based on Textual Information 8 Dec 2023 · 0 repositories · arXiv:2312.04944
-
Contrastive Feature Masking Open-Vocabulary Vision Transformer 2 Sep 2023 · 0 repositories · arXiv:2309.00775
-
Learning Large Graph Property Prediction via Graph Segment Training 21 May 2023 · 1 repository · arXiv:2305.12322Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 2 violated, 1 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples) · 4 pointer-only (licence)
-
Explainable and High-Performance Hate and Offensive Speech Detection 26 Jun 2022 · 0 repositories · arXiv:2206.12983
-
IIITT@Dravidian-CodeMix-FIRE2021: Transliterate or translate? Sentiment analysis of code-mixed text in Dravidian languages 15 Nov 2021 · 1 repository · arXiv:2111.07906
-
Offensive Language Identification in Low-resourced Code-mixed Dravidian languages using Pseudo-labeling 27 Aug 2021 · 1 repository · arXiv:2108.12177
-
Towards Offensive Language Identification for Tamil Code-Mixed YouTube Comments and Posts 24 Aug 2021 · 1 repository · arXiv:2108.10939
-
SHAQ: Single Headed Attention with Quasi-Recurrence 18 Aug 2021 · 0 repositories · arXiv:2108.08207
-
Learning ULMFiT and Self-Distillation with Calibration for Medical Dialogue System 20 Jul 2021 · 0 repositories · arXiv:2107.09625
-
Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning 4 Jun 2021 · 3 repositories · arXiv:2106.02584Syntology official (archive's flag): 6 ran · 22 ran (of which 4 constructed an object rather than computing a result; 20 with no instrument failure: 2 honoured, 1 violated, 17 with no contract checked; 2 where Syntology's instrument failed) · 8 unverified (of 30 harvested samples)
-
WHOSe Heritage: Classification of UNESCO World Heritage "Outstanding Universal Value" Documents with Soft Labels 12 Apr 2021 · 1 repository · arXiv:2104.05547
-
L3CubeMahaSent: A Marathi Tweet-based Sentiment Analysis Dataset 21 Mar 2021 · 1 repository · arXiv:2103.11408
-
indicnlp@kgp at DravidianLangTech-EACL2021: Offensive Language Identification in Dravidian Languages 14 Feb 2021 · 1 repository · arXiv:2102.07150
-
indicnlp@ kgp at DravidianLangTech-EACL2021: Offensive Language Identification in Dravidian Languages 14 Feb 2021 · 1 repository
-
Train your classifier first: Cascade Neural Networks Training from upper layers to lower layers 9 Feb 2021 · 0 repositories · arXiv:2102.04697
-
Experimental Evaluation of Deep Learning models for Marathi Text Classification 13 Jan 2021 · 0 repositories · arXiv:2101.04899
-
LaDiff ULMFiT: A Layer Differentiated training approach for ULMFiT 13 Jan 2021 · 1 repository · arXiv:2101.04965
-
Post-Training Weighted Quantization of Neural Networks for Language Models 1 Jan 2021 · 0 repositories
-
HinglishNLP at SemEval-2020 Task 9: Fine-tuned Language Models for Hinglish Sentiment Detection 1 Dec 2020 · 2 repositories
-
Smash at SemEval-2020 Task 7: Optimizing the Hyperparameters of ERNIE 2.0 for Humor Ranking and Rating 1 Dec 2020 · 0 repositories
-
Palomino-Ochoa at SemEval-2020 Task 9: Robust System based on Transformer for Code-Mixed Sentiment Classification 18 Nov 2020 · 0 repositories · arXiv:2011.09448
-
Pagsusuri ng RNN-based Transfer Learning Technique sa Low-Resource Language 13 Oct 2020 · 2 repositories · arXiv:2010.06447
-
Gauravarora@HASOC-Dravidian-CodeMix-FIRE2020: Pre-training ULMFiT on Synthetically Generated Code-Mixed Data for Hate Speech Detection 5 Oct 2020 · 0 repositories · arXiv:2010.02094
-
Fine-tuning Pre-trained Contextual Embeddings for Citation Content Analysis in Scholarly Publication 12 Sep 2020 · 0 repositories · arXiv:2009.05836
-
HinglishNLP: Fine-tuned Language Models for Hinglish Sentiment Detection 22 Aug 2020 · 2 repositories · arXiv:2008.09820
-
Composer Style Classification of Piano Sheet Music Images Using Language Model Pretraining 29 Jul 2020 · 1 repository · arXiv:2007.14587
-
ECPE-2D: Emotion-Cause Pair Extraction based on Joint Two-Dimensional Representation, Interaction and Prediction 1 Jul 2020 · 1 repository
-
Probing for Referential Information in Language Models 1 Jul 2020 · 0 repositories
-
Evaluation Metrics for Headline Generation Using Deep Pre-Trained Embeddings 1 May 2020 · 0 repositories
-
Offensive language detection in Arabic using ULMFiT 1 May 2020 · 1 repository
-
Text Categorization for Conflict Event Annotation 1 May 2020 · 0 repositories
-
Inferring the source of official texts: can SVM beat ULMFiT? 2 Mar 2020 · 1 repository
-
MaxUp: A Simple Way to Improve Generalization of Neural Network Training 20 Feb 2020 · 1 repository · arXiv:2002.09024Syntology 0 ran · 1 unverified (of 1 harvested sample)
-
Localized Flood DetectionWith Minimal Labeled Social Media Data Using Transfer Learning 10 Feb 2020 · 0 repositories · arXiv:2003.04973
-
Natural language processing of MIMIC-III clinical notes for identifying diagnosis and procedures with neural networks 28 Dec 2019 · 0 repositories · arXiv:1912.12397
-
A Comparative Study of Pretrained Language Models on Thai Social Text Categorization 3 Dec 2019 · 0 repositories · arXiv:1912.01580
-
DeFINE: DEep Factorized INput Token Embeddings for Neural Sequence Modeling 27 Nov 2019 · 1 repository · arXiv:1911.12385
-
Single Headed Attention RNN: Stop Thinking With Your Head 26 Nov 2019 · 5 repositories · arXiv:1911.11423
-
A Subword Level Language Model for Bangla Language 15 Nov 2019 · 0 repositories · arXiv:1911.07613
-
Improving Transformer Models by Reordering their Sublayers 10 Nov 2019 · 2 repositories · arXiv:1911.03864
-
Evolution of transfer learning in natural language processing 16 Oct 2019 · 0 repositories · arXiv:1910.07370
-
The merits of Universal Language Model Fine-tuning for Small Datasets -- a case with Dutch book reviews 2 Oct 2019 · 2 repositories · arXiv:1910.00896
-
Analyzing Customer Feedback for Product Fit Prediction 28 Aug 2019 · 0 repositories · arXiv:1908.10896
-
Low-Shot Classification: A Comparison of Classical and Deep Transfer Machine Learning Approaches 17 Jul 2019 · 0 repositories · arXiv:1907.07543
-
Evaluating Language Model Finetuning Techniques for Low-resource Languages 30 Jun 2019 · 2 repositories · arXiv:1907.00409
-
Exploiting Unsupervised Pre-training and Automated Feature Engineering for Low-resource Hate Speech Detection in Polish 17 Jun 2019 · 0 repositories · arXiv:1906.09325
-
Figure Eight at SemEval-2019 Task 3: Ensemble of Transfer Learning Methods for Contextual Emotion Detection 1 Jun 2019 · 0 repositories
-
Speak up, Fight Back! Detection of Social Media Disclosures of Sexual Harassment 1 Jun 2019 · 0 repositories
-
Adaptive Attention Span in Transformers 19 May 2019 · 8 repositories · arXiv:1905.07799
-
An Empirical Evaluation of Text Representation Schemes on Multilingual Social Web to Filter the Textual Aggression 16 Apr 2019 · 0 repositories · arXiv:1904.08770
-
Low Resource Text Classification with ULMFit and Backtranslation 21 Mar 2019 · 1 repository · arXiv:1903.09244
-
Classification of Medication-Related Tweets Using Stacked Bidirectional LSTMs with Context-Aware Attention 1 Oct 2018 · 1 repository
-
Online Cyber-Attack Detection in Smart Grid: A Reinforcement Learning Approach 14 Sep 2018 · 1 repository · arXiv:1809.05258
-
Language Informed Modeling of Code-Switched Text 1 Jul 2018 · 0 repositories
-
Universal Language Model Fine-tuning for Text Classification 18 Jan 2018 · 66 repositories · arXiv:1801.06146Syntology community repositories only · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (of 5 harvested samples) · 3 pointer-only (licence)
-
Breaking the Softmax Bottleneck: A High-Rank RNN Language Model 10 Nov 2017 · 9 repositories · arXiv:1711.03953Syntology official (archive's flag): 2 ran · 12 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 0 violated, 11 with no contract checked; 1 where Syntology's instrument failed) · 11 unverified (of 23 harvested samples) · 1 pointer-only (licence)
-
Regularizing and Optimizing LSTM Language Models 7 Aug 2017 · 45 repositories · arXiv:1708.02182Syntology community repositories only · 6 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples) · 7 pointer-only (licence)
-
A Theoretically Grounded Application of Dropout in Recurrent Neural Networks 16 Dec 2015 · 14 repositories · arXiv:1512.05287Syntology 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
Tasks archive 2025-07-28
20 shown of 85 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections