Methods › General › Regularization › Variational Dropout
Variational Dropout
Introduced by Yarin Gal et al. in A Theoretically Grounded Application of Dropout in Recurrent Neural Networks
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Variational Dropout is a regularization technique based on dropout, but uses a variational inference grounded approach. In Variational Dropout, we repeat the same dropout mask at each time step for both inputs, outputs, and recurrent layers (drop the same network units at each time step). This is in contrast to ordinary Dropout where different dropout masks are sampled at each time step for the inputs and outputs alone.
In Syntology name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.
Near matches the lookup lists, each a different name: Meta-Variational Dropout (MetaVD); Neural Variational Dropout Processes; Rep-Dropout; Loop Dropout; late dropout; DROPOUT-GRPO; early dropout; Rubric Dropout.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 141 · with a code link: 73 · where Syntology ran a sample: 22 (20 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (22 of 141 tagged: 20 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 141, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
RLBenchNet: The Right Network for the Right Reinforcement Learning Task 21 May 2025 · 1 repository · arXiv:2505.15040
-
Advanced Deep Learning Techniques for Analyzing Earnings Call Transcripts: Methodologies and Applications 27 Feb 2025 · 0 repositories · arXiv:2503.01886
-
BARNN: A Bayesian Autoregressive and Recurrent Neural Network 30 Jan 2025 · 0 repositories · arXiv:2501.18665
-
A Combined Encoder and Transformer Approach for Coherent and High-Quality Text Generation 19 Nov 2024 · 0 repositories · arXiv:2411.12157
-
No Argument Left Behind: Overlapping Chunks for Faster Processing of Arbitrarily Long Legal Texts 24 Oct 2024 · 0 repositories · arXiv:2410.19184
-
Large Body Language Models 21 Oct 2024 · 0 repositories · arXiv:2410.16533
-
RICo: Reddit ideological communities 5 Jun 2024 · 1 repository
-
Transformers for Supervised Online Continual Learning 3 Mar 2024 · 0 repositories · arXiv:2403.01554
-
UniMem: Towards a Unified View of Long-Context Large Language Models 5 Feb 2024 · 1 repository · arXiv:2402.03009
-
Exploring Multi-Level Threats in Telegram Data with AI-Human Annotation: A Preliminary Study 15 Dec 2023 · 0 repositories
-
Illicit Darkweb Classification via Natural-language Processing: Classifying Illicit Content of Webpages based on Textual Information 8 Dec 2023 · 0 repositories · arXiv:2312.04944
-
Memory-efficient Stochastic methods for Memory-based Transformers 14 Nov 2023 · 1 repository · arXiv:2311.08123
-
TRAMS: Training-free Memory Selection for Long-range Language Modeling 24 Oct 2023 · 1 repository · arXiv:2310.15494Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
Approximating Two-Layer Feedforward Networks for Efficient Transformers 16 Oct 2023 · 2 repositories · arXiv:2310.10837Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples)
-
Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agents 29 Sep 2023 · 1 repository · arXiv:2309.17207Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
Random-Access Infinite Context Length for Transformers 21 Sep 2023 · 1 repository
-
RCMHA: Relative Convolutional Multi-Head Attention for Natural Language Modelling 7 Aug 2023 · 1 repository · arXiv:2308.03429
-
Landmark Attention: Random-Access Infinite Context Length for Transformers 25 May 2023 · 2 repositories · arXiv:2305.16300Syntology official (archive's flag): 1 ran · 11 ran (of which 5 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 6 where Syntology's instrument failed) · 2 unverified (of 13 harvested samples)
-
Sparsified Model Zoo Twins: Investigating Populations of Sparsified Neural Network Models 26 Apr 2023 · 0 repositories · arXiv:2304.13718
-
Transformer-based World Models Are Happy With 100k Interactions 13 Mar 2023 · 1 repository · arXiv:2303.07109Syntology official (archive's flag): 16 ran · 16 ran (of which 6 constructed an object rather than computing a result; 8 with no instrument failure: 1 honoured, 1 violated, 6 with no contract checked; 8 where Syntology's instrument failed) · 9 unverified (of 25 harvested samples)
-
GTR-CTRL: Instrument and Genre Conditioning for Guitar-Focused Music Generation with Transformers 10 Feb 2023 · 0 repositories · arXiv:2302.05393
-
An Comparative Analysis of Different Pitch and Metrical Grid Encoding Methods in the Task of Sequential Music Generation 31 Jan 2023 · 0 repositories · arXiv:2301.13383
-
Efficient Sparsely Activated Transformers 31 Aug 2022 · 0 repositories · arXiv:2208.14580
-
Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models 13 Aug 2022 · 9 repositories · arXiv:2208.06677Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
Recurrent Memory Transformer 14 Jul 2022 · 3 repositories · arXiv:2207.06881Syntology official (archive's flag): 3 ran · 6 ran (of which 3 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 1 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 7 unverified (of 13 harvested samples) · 2 pointer-only (licence)
-
Explainable and High-Performance Hate and Offensive Speech Detection 26 Jun 2022 · 0 repositories · arXiv:2206.12983
-
Emotion-Aware Transformer Encoder for Empathetic Dialogue Generation 24 Apr 2022 · 1 repository · arXiv:2204.11320
-
SinTra: Learning an inspiration model from a single multi-track music segment 21 Apr 2022 · 1 repository · arXiv:2204.09917
-
LiteTransformerSearch: Training-free Neural Architecture Search for Efficient Language Models 4 Mar 2022 · 1 repository · arXiv:2203.02094Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 1 pointer-only (licence)
-
Reconsidering the Past: Optimizing Hidden States in Language Models 16 Dec 2021 · 0 repositories · arXiv:2112.08653
-
A Comparative Study of Transformers on Word Sense Disambiguation 30 Nov 2021 · 0 repositories · arXiv:2111.15417
-
How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN 18 Nov 2021 · 0 repositories · arXiv:2111.09509
-
IIITT@Dravidian-CodeMix-FIRE2021: Transliterate or translate? Sentiment analysis of code-mixed text in Dravidian languages 15 Nov 2021 · 1 repository · arXiv:2111.07906
-
TACCL: Guiding Collective Algorithm Synthesis using Communication Sketches 8 Nov 2021 · 2 repositories · arXiv:2111.04867
-
Language Modelling via Learning to Rank 13 Oct 2021 · 0 repositories · arXiv:2110.06961
-
A study of the robustness of raw waveform based speaker embeddings under mismatched conditions 8 Oct 2021 · 1 repository · arXiv:2110.04265
-
Layer-wise Pruning of Transformer Attention Heads for Efficient Language Modeling 7 Oct 2021 · 1 repository · arXiv:2110.03252
-
Neural Variational Dropout Processes 29 Sep 2021 · 0 repositories
-
Offensive Language Identification in Low-resourced Code-mixed Dravidian languages using Pseudo-labeling 27 Aug 2021 · 1 repository · arXiv:2108.12177
-
Towards Offensive Language Identification for Tamil Code-Mixed YouTube Comments and Posts 24 Aug 2021 · 1 repository · arXiv:2108.10939
-
Learning Sparse Analytic Filters for Piano Transcription 23 Aug 2021 · 2 repositories · arXiv:2108.10382
-
FNetAR: Mixing Tokens with Autoregressive Fourier Transforms 22 Jul 2021 · 1 repository · arXiv:2107.10932
-
Learning ULMFiT and Self-Distillation with Calibration for Medical Dialogue System 20 Jul 2021 · 0 repositories · arXiv:2107.09625
-
Transformers with multi-modal features and post-fusion context for e-commerce session-based recommendation 11 Jul 2021 · 0 repositories · arXiv:2107.05124
-
ASR Adaptation for E-commerce Chatbots using Cross-Utterance Context and Multi-Task Language Modeling 15 Jun 2021 · 0 repositories · arXiv:2106.09532
-
The Flip Side of the Reweighted Coin: Duality of Adaptive Dropout and Regularization 14 Jun 2021 · 1 repository · arXiv:2106.07769Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
Towards a Better Tradeoff between Effectiveness and Efficiency in Pre-Ranking: A Learnable Feature Selection based Approach 17 May 2021 · 0 repositories · arXiv:2105.07706
-
Adapting Long Context NLM for ASR Rescoring in Conversational Agents 21 Apr 2021 · 0 repositories · arXiv:2104.11070
-
WHOSe Heritage: Classification of UNESCO World Heritage "Outstanding Universal Value" Documents with Soft Labels 12 Apr 2021 · 1 repository · arXiv:2104.05547
-
FastMoE: A Fast Mixture-of-Expert Training System 24 Mar 2021 · 3 repositories · arXiv:2103.13262Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples) · 1 pointer-only (licence)
-
L3CubeMahaSent: A Marathi Tweet-based Sentiment Analysis Dataset 21 Mar 2021 · 1 repository · arXiv:2103.11408
-
Structured Dropout Variational Inference for Bayesian Neural Networks 16 Feb 2021 · 0 repositories · arXiv:2102.07927
-
indicnlp@kgp at DravidianLangTech-EACL2021: Offensive Language Identification in Dravidian Languages 14 Feb 2021 · 1 repository · arXiv:2102.07150
-
indicnlp@ kgp at DravidianLangTech-EACL2021: Offensive Language Identification in Dravidian Languages 14 Feb 2021 · 1 repository
-
Train your classifier first: Cascade Neural Networks Training from upper layers to lower layers 9 Feb 2021 · 0 repositories · arXiv:2102.04697
-
Adaptive Semiparametric Language Models 4 Feb 2021 · 0 repositories · arXiv:2102.02557
-
Mind the Gap: Assessing Temporal Generalization in Neural Language Models 3 Feb 2021 · 1 repository · arXiv:2102.01951
-
Experimental Evaluation of Deep Learning models for Marathi Text Classification 13 Jan 2021 · 0 repositories · arXiv:2101.04899
-
LaDiff ULMFiT: A Layer Differentiated training approach for ULMFiT 13 Jan 2021 · 1 repository · arXiv:2101.04965
-
Post-Training Weighted Quantization of Neural Networks for Language Models 1 Jan 2021 · 0 repositories
-
Subformer: A Parameter Reduced Transformer 1 Jan 2021 · 0 repositories
-
HinglishNLP at SemEval-2020 Task 9: Fine-tuned Language Models for Hinglish Sentiment Detection 1 Dec 2020 · 2 repositories
-
Hitachi at SemEval-2020 Task 8: Simple but Effective Modality Ensemble for Meme Emotion Recognition 1 Dec 2020 · 0 repositories
-
Smash at SemEval-2020 Task 7: Optimizing the Hyperparameters of ERNIE 2.0 for Humor Ranking and Rating 1 Dec 2020 · 0 repositories
-
Palomino-Ochoa at SemEval-2020 Task 9: Robust System based on Transformer for Code-Mixed Sentiment Classification 18 Nov 2020 · 0 repositories · arXiv:2011.09448
-
Improving RNN transducer with normalized jointer network 3 Nov 2020 · 0 repositories · arXiv:2011.01576
-
Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Dataset 22 Oct 2020 · 0 repositories · arXiv:2010.11395
-
Transferable Graph Optimizers for ML Compilers 21 Oct 2020 · 0 repositories · arXiv:2010.12438
-
Memformer: A Memory-Augmented Transformer for Sequence Modeling 14 Oct 2020 · 0 repositories · arXiv:2010.06891
-
Pagsusuri ng RNN-based Transfer Learning Technique sa Low-Resource Language 13 Oct 2020 · 2 repositories · arXiv:2010.06447
-
Gauravarora@HASOC-Dravidian-CodeMix-FIRE2020: Pre-training ULMFiT on Synthetically Generated Code-Mixed Data for Hate Speech Detection 5 Oct 2020 · 0 repositories · arXiv:2010.02094
-
Fine-tuning Pre-trained Contextual Embeddings for Citation Content Analysis in Scholarly Publication 12 Sep 2020 · 0 repositories · arXiv:2009.05836
-
Pay Attention when Required 9 Sep 2020 · 2 repositories · arXiv:2009.04534
-
HinglishNLP: Fine-tuned Language Models for Hinglish Sentiment Detection 22 Aug 2020 · 2 repositories · arXiv:2008.09820
-
The Jazz Transformer on the Front Line: Exploring the Shortcomings of AI-composed Music through Quantitative Measures 4 Aug 2020 · 2 repositories · arXiv:2008.01307Syntology official (archive's flag): 13 ran · 13 ran (of which 0 constructed an object rather than computing a result; 13 with no instrument failure: 0 honoured, 0 violated, 13 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 14 harvested samples)
-
Language Modelling for Source Code with Transformer-XL 31 Jul 2020 · 1 repository · arXiv:2007.15813
-
Composer Style Classification of Piano Sheet Music Images Using Language Model Pretraining 29 Jul 2020 · 1 repository · arXiv:2007.14587
-
VINNAS: Variational Inference-based Neural Network Architecture Search 12 Jul 2020 · 0 repositories · arXiv:2007.06103
-
Do Transformers Need Deep Long-Range Memory 7 Jul 2020 · 1 repository · arXiv:2007.03356
-
Do Transformers Need Deep Long-Range Memory? 1 Jul 2020 · 0 repositories
-
Probing for Referential Information in Language Models 1 Jul 2020 · 0 repositories
-
Mind The Facts: Knowledge-Boosted Coherent Abstractive Text Summarization 27 Jun 2020 · 0 repositories · arXiv:2006.15435
-
Exploring Transformers for Large-Scale Speech Recognition 19 May 2020 · 0 repositories · arXiv:2005.09684
-
Evaluation Metrics for Headline Generation Using Deep Pre-Trained Embeddings 1 May 2020 · 0 repositories
-
Improving Neural Language Generation with Spectrum Control 1 May 2020 · 0 repositories
-
Offensive language detection in Arabic using ULMFiT 1 May 2020 · 1 repository
-
Text Categorization for Conflict Event Annotation 1 May 2020 · 0 repositories
-
Segatron: Segment-Aware Transformer for Language Modeling and Understanding 30 Apr 2020 · 1 repository · arXiv:2004.14996
-
Bayesian Sparsification Methods for Deep Complex-valued Networks 25 Mar 2020 · 1 repository · arXiv:2003.11413
-
Finnish Language Modeling with Deep Transformer Models 14 Mar 2020 · 0 repositories · arXiv:2003.11562
-
Adaptive Neural Connections for Sparsity Learning 5 Mar 2020 · 0 repositories
-
Inferring the source of official texts: can SVM beat ULMFiT? 2 Mar 2020 · 1 repository
-
MaxUp: A Simple Way to Improve Generalization of Neural Network Training 20 Feb 2020 · 1 repository · arXiv:2002.09024Syntology 0 ran · 1 unverified (of 1 harvested sample)
-
Localized Flood DetectionWith Minimal Labeled Social Media Data Using Transfer Learning 10 Feb 2020 · 0 repositories · arXiv:2003.04973
-
Variational Dropout Sparsification for Particle Identification speed-up 21 Jan 2020 · 1 repository · arXiv:2001.07493
-
Delving Deeper into the Decoder for Video Captioning 16 Jan 2020 · 1 repository · arXiv:2001.05614Syntology official: harvested, nothing ran · 0 ran · 2 unverified (of 2 harvested samples)
-
Natural language processing of MIMIC-III clinical notes for identifying diagnosis and procedures with neural networks 28 Dec 2019 · 0 repositories · arXiv:1912.12397
-
A Comparative Study of Pretrained Language Models on Thai Social Text Categorization 3 Dec 2019 · 0 repositories · arXiv:1912.01580
-
Neural Academic Paper Generation 2 Dec 2019 · 1 repository · arXiv:1912.01982
-
DeFINE: DEep Factorized INput Token Embeddings for Neural Sequence Modeling 27 Nov 2019 · 1 repository · arXiv:1911.12385
Tasks archive 2025-07-28
20 shown of 144 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections