Methods › General › Learning Rate Schedules › Cosine Annealing › Papers, page 40

Cosine Annealing

Papers archive 2025-07-28

archive papers tagged: 3,965 · with a code link: 1,734 · where Syntology ran a sample: 627 (513 with a run with no instrument failure, 114 where every run was a failure of Syntology's instrument) Syntology

Show: all tagged papersonly where code ran (627 of 3,965 tagged: 513 with a run with no instrument failure, 114 where every run was a failure of Syntology's instrument)

Page 40 of 40: papers 3,901 to 3,965 of 3,965, newest first by the archive's date (ties by slug), in archive order.

Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.

• Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
  17 Sep 2019 · 10 repositories · arXiv:1909.08053
  Syntology community repositories only · 12 ran (of which 2 constructed an object rather than computing a result; 7 with no instrument failure: 4 honoured, 0 violated, 3 with no contract checked; 5 where Syntology's instrument failed) · 35 unverified (of 47 harvested samples) · 15 pointer-only (licence)
• A Self-Attentional Neural Architecture for Code Completion with Multi-Task Learning
  16 Sep 2019 · 0 repositories · arXiv:1909.06983
• Ouroboros: On Accelerating Training of Transformer-Based Language Models
  14 Sep 2019 · 1 repository · arXiv:1909.06695
• Reasoning Over Semantic-Level Graph for Fact Checking
  9 Sep 2019 · 0 repositories · arXiv:1909.03745
• Effective Use of Transformer Networks for Entity Tracking
  5 Sep 2019 · 1 repository · arXiv:1909.02635
  Syntology official (archive's flag): 1 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 5 harvested samples) · 5 pointer-only (licence)
• Semantics-aware BERT for Language Understanding
  5 Sep 2019 · 1 repository · arXiv:1909.02209
  Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 5 unverified (of 12 harvested samples) · 3 pointer-only (licence)
• How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings
  2 Sep 2019 · 1 repository · arXiv:1909.00512
• Adversarial Learning with Contextual Embeddings for Zero-resource Cross-lingual Classification and NER
  31 Aug 2019 · 0 repositories · arXiv:1909.00153
• Quantity doesn't buy quality syntax with neural language models
  31 Aug 2019 · 0 repositories · arXiv:1909.00111
• Adaptively Sparse Transformers
  30 Aug 2019 · 3 repositories · arXiv:1909.00015
  Syntology official (archive's flag): 2 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
• Pre-training A Neural Language Model Improves The Sample Efficiency of an Emergency Room Classification Model
  30 Aug 2019 · 0 repositories · arXiv:1909.01136
• Measuring Patent Claim Generation by Span Relevancy
  26 Aug 2019 · 0 repositories · arXiv:1908.09591
• Release Strategies and the Social Impacts of Language Models
  24 Aug 2019 · 0 repositories · arXiv:1908.09203
• Mish: A Self Regularized Non-Monotonic Activation Function
  23 Aug 2019 · 9 repositories · arXiv:1908.08681
  Syntology official (archive's flag): 6 ran · 9 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 1 honoured, 0 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (of 12 harvested samples) · 1 pointer-only (licence)
• Universal Adversarial Triggers for Attacking and Analyzing NLP
  20 Aug 2019 · 1 repository · arXiv:1908.07125
  Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
• SCARLET-NAS: Bridging the Gap between Stability and Scalability in Weight-sharing Neural Architecture Search
  16 Aug 2019 · 1 repository · arXiv:1908.06022
• BioFLAIR: Pretrained Pooled Contextualized Embeddings for Biomedical Sequence Labeling Tasks
  13 Aug 2019 · 1 repository · arXiv:1908.05760
• Attentive Normalization
  4 Aug 2019 · 2 repositories · arXiv:1908.01259
• MoGA: Searching Beyond MobileNetV3
  4 Aug 2019 · 2 repositories · arXiv:1908.01314
• Noisy Channel for Low Resource Grammatical Error Correction
  1 Aug 2019 · 0 repositories
• Leveraging Pre-trained Checkpoints for Sequence Generation Tasks
  29 Jul 2019 · 7 repositories · arXiv:1907.12461
  Syntology 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples)
• DLGNet: A Transformer-based Model for Dialogue Response Generation
  26 Jul 2019 · 0 repositories · arXiv:1908.01841
• Generating Sentiment-Preserving Fake Online Reviews Using Neural Language Models and Their Human- and Machine-based Detection
  22 Jul 2019 · 0 repositories · arXiv:1907.09177
• Patent Claim Generation by Fine-Tuning OpenAI GPT-2
  1 Jul 2019 · 0 repositories · arXiv:1907.02052
• GPT-based Generation for Classical Chinese Poetry
  29 Jun 2019 · 3 repositories · arXiv:1907.00151
• A Tensorized Transformer for Language Modeling
  24 Jun 2019 · 1 repository · arXiv:1906.09777
  Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample) · 1 pointer-only (licence)
• Densely Connected Search Space for More Flexible Neural Architecture Search
  23 Jun 2019 · 1 repository · arXiv:1906.09607
  Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)
• Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction
  19 Jun 2019 · 1 repository · arXiv:1906.08646
• XLNet: Generalized Autoregressive Pretraining for Language Understanding
  19 Jun 2019 · 27 repositories · arXiv:1906.08237
  Syntology official (archive's flag): 1 ran · 15 ran (of which 2 constructed an object rather than computing a result; 10 with no instrument failure: 2 honoured, 0 violated, 8 with no contract checked; 5 where Syntology's instrument failed) · 9 unverified (of 24 harvested samples) · 4 pointer-only (licence)
• XNAS: Neural Architecture Search with Expert Advice
  19 Jun 2019 · 2 repositories · arXiv:1906.08031
• One Epoch Is All You Need
  16 Jun 2019 · 0 repositories · arXiv:1906.06669
• A Multiscale Visualization of Attention in the Transformer Model
  12 Jun 2019 · 3 repositories · arXiv:1906.05714
  Syntology community repositories only · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
• Analyzing the Structure of Attention in a Transformer Language Model
  7 Jun 2019 · 0 repositories · arXiv:1906.04284
• CODAH: An Adversarially-Authored Question Answering Dataset for Common Sense
  1 Jun 2019 · 1 repository
• Figure Eight at SemEval-2019 Task 3: Ensemble of Transfer Learning Methods for Contextual Emotion Detection
  1 Jun 2019 · 0 repositories
• Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)
  28 May 2019 · 1 repository · arXiv:1905.11833
  Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample)
• Story Ending Prediction by Transferable BERT
  17 May 2019 · 1 repository · arXiv:1905.07504
• Transformer-XL: Language Modeling with Longer-Term Dependency
  1 May 2019 · 0 repositories
• GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond
  25 Apr 2019 · 9 repositories · arXiv:1904.11492
• Generating Long Sequences with Sparse Transformers
  23 Apr 2019 · 7 repositories · arXiv:1904.10509
  Syntology community repositories only · 5 ran (of which 4 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples)
• Attention Augmented Convolutional Networks
  22 Apr 2019 · 14 repositories · arXiv:1904.09925
  Syntology 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (of 6 harvested samples) · 1 pointer-only (licence)
• Neural Architecture Search for Deep Face Recognition
  21 Apr 2019 · 0 repositories · arXiv:1904.09523
• Language Models with Transformers
  20 Apr 2019 · 1 repository · arXiv:1904.09408
  Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
• Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution
  10 Apr 2019 · 28 repositories · arXiv:1904.05049
  Syntology community repositories only · 24 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 1 honoured, 0 violated, 10 with no contract checked; 13 where Syntology's instrument failed) · 10 unverified (of 34 harvested samples) · 9 pointer-only (licence)
• NLPR@SRPOL at SemEval-2019 Task 6 and Task 5: Linguistically enhanced deep learning offensive sentence classifier
  10 Apr 2019 · 0 repositories · arXiv:1904.05152
• CondConv: Conditionally Parameterized Convolutions for Efficient Inference
  10 Apr 2019 · 9 repositories · arXiv:1904.04971
• ASAP: Architecture Search, Anneal and Prune
  8 Apr 2019 · 1 repository · arXiv:1904.04123
• Visualizing Attention in Transformer-Based Language Representation Models
  4 Apr 2019 · 0 repositories · arXiv:1904.02679
• Exploring Randomly Wired Neural Networks for Image Recognition
  2 Apr 2019 · 9 repositories · arXiv:1904.01569
  Syntology 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 21 unverified (of 31 harvested samples) · 1 pointer-only (licence)
• Distilling Task-Specific Knowledge from BERT into Simple Neural Networks
  28 Mar 2019 · 4 repositories · arXiv:1903.12136
  Syntology 9 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 3 honoured, 0 violated, 0 with no contract checked; 6 where Syntology's instrument failed) · 0 unverified (of 9 harvested samples)
• Language Models are Unsupervised Multitask Learners
  14 Feb 2019 · 21 repositories
• Passage Re-ranking with BERT
  13 Jan 2019 · 6 repositories · arXiv:1901.04085
  Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 13 harvested samples) · 2 pointer-only (licence)
• Linguistic Analysis of Pretrained Sentence Encoders with Acceptability Judgments
  11 Jan 2019 · 0 repositories · arXiv:1901.03438
• Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
  9 Jan 2019 · 37 repositories · arXiv:1901.02860
  Syntology official (archive's flag): 8 ran · 67 ran (of which 37 constructed an object rather than computing a result; 49 with no instrument failure: 4 honoured, 1 violated, 44 with no contract checked; 18 where Syntology's instrument failed) · 76 unverified (of 143 harvested samples) · 43 pointer-only (licence)
• FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search
  9 Dec 2018 · 5 repositories · arXiv:1812.03443
  Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
• Bag of Tricks for Image Classification with Convolutional Neural Networks
  4 Dec 2018 · 28 repositories · arXiv:1812.01187
  Syntology community repositories only · 11 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 2 honoured, 0 violated, 7 with no contract checked; 2 where Syntology's instrument failed) · 4 unverified (of 15 harvested samples) · 5 pointer-only (licence)
• Pelee: A Real-Time Object Detection System on Mobile Devices
  1 Dec 2018 · 2 repositories
• A Closer Look at Deep Learning Heuristics: Learning rate restarts, Warmup and Distillation
  29 Oct 2018 · 0 repositories · arXiv:1810.13243
• Big-Little Net: An Efficient Multi-Scale Feature Representation for Visual and Speech Recognition
  10 Jul 2018 · 3 repositories · arXiv:1807.03848
• Using Mode Connectivity for Loss Landscape Analysis
  18 Jun 2018 · 0 repositories · arXiv:1806.06977
• Improving Language Understanding by Generative Pre-Training
  11 Jun 2018 · 13 repositories
• Averaging Weights Leads to Wider Optima and Better Generalization
  14 Mar 2018 · 17 repositories · arXiv:1803.05407
  Syntology official (archive's flag): 2 ran · 7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 1 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 9 harvested samples) · 4 pointer-only (licence)
• Regularized Evolution for Image Classifier Architecture Search
  5 Feb 2018 · 5 repositories · arXiv:1802.01548
• From Facial Expression Recognition to Interpersonal Relation Prediction
  21 Sep 2016 · 0 repositories · arXiv:1609.06426
• SGDR: Stochastic Gradient Descent with Warm Restarts
  13 Aug 2016 · 24 repositories · arXiv:1608.03983
  Syntology community repositories only · 16 ran (of which 7 constructed an object rather than computing a result; 15 with no instrument failure: 0 honoured, 0 violated, 15 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 17 harvested samples) · 7 pointer-only (licence)