Methods › General › Attention Modules › Multi-Head Attention › Papers where code ran, page 35

Multi-Head Attention

Papers archive 2025-07-28

archive papers tagged: 24,855 · with a code link: 11,214 · where Syntology ran a sample: 3,454 (2,916 with a run with no instrument failure, 538 where every run was a failure of Syntology's instrument) Syntology

Show: all tagged papersonly where code ran (3,454 of 24,855 tagged: 2,916 with a run with no instrument failure, 538 where every run was a failure of Syntology's instrument)

Syntology We ran code from the paper's repository; we did not isolate this method inside it.

Page 35 of 35: papers 3,401 to 3,454 of the 3,454 tagged papers where Syntology ran at least one harvested sample (2,916 with a run with no instrument failure, 538 where every run was a failure of Syntology's instrument), newest first by the archive's date (ties by arXiv id). This is a filter on Syntology's record ordered by date only, not a ranking; a run is not a correctness claim. A paper missing from this list is not a recorded non-run: it may have no arXiv id, no harvested code, or only samples that have not run yet.

Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.

• Generating Long Sequences with Sparse Transformers
  23 Apr 2019 · 7 repositories · arXiv:1904.10509
  Syntology community repositories only · 5 ran (of which 4 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples)
• Attention Augmented Convolutional Networks
  22 Apr 2019 · 14 repositories · arXiv:1904.09925
  Syntology 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (of 6 harvested samples) · 1 pointer-only (licence)
• Language Models with Transformers
  20 Apr 2019 · 1 repository · arXiv:1904.09408
  Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
• ERNIE: Enhanced Representation through Knowledge Integration
  19 Apr 2019 · 19 repositories · arXiv:1904.09223
  Syntology community repositories only · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples)
• Mask-Predict: Parallel Decoding of Conditional Masked Language Models
  19 Apr 2019 · 2 repositories · arXiv:1904.09324
  Syntology community repositories only · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
• DocBERT: BERT for Document Classification
  17 Apr 2019 · 3 repositories · arXiv:1904.08398
  Syntology official (archive's flag): 6 ran · 11 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 0 violated, 11 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 14 harvested samples)
• CEDR: Contextualized Embeddings for Document Ranking
  15 Apr 2019 · 7 repositories · arXiv:1904.07094
  Syntology official (archive's flag): 4 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples) · 2 pointer-only (licence)
• Rare Words: A Major Problem for Contextualized Embeddings And How to Fix it by Attentive Mimicking
  14 Apr 2019 · 2 repositories · arXiv:1904.06707
  Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
• Simple BERT Models for Relation Extraction and Semantic Role Labeling
  10 Apr 2019 · 3 repositories · arXiv:1904.05255
  Syntology community repositories only · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
• Jointly Measuring Diversity and Quality in Text Generation Models
  8 Apr 2019 · 3 repositories · arXiv:1904.03971
  Syntology official (archive's flag): 1 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (of 6 harvested samples) · 1 pointer-only (licence)
• Publicly Available Clinical BERT Embeddings
  6 Apr 2019 · 3 repositories · arXiv:1904.03323
  Syntology official (archive's flag): 1 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
• VideoBERT: A Joint Model for Video and Language Representation Learning
  3 Apr 2019 · 3 repositories · arXiv:1904.01766
  Syntology 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
• 75 Languages, 1 Model: Parsing Universal Dependencies Universally
  3 Apr 2019 · 3 repositories · arXiv:1904.02099
  Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 12 harvested samples)
• BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis
  3 Apr 2019 · 1 repository · arXiv:1904.02232
  Syntology 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (of 7 harvested samples) · 1 pointer-only (licence)
• Large Batch Optimization for Deep Learning: Training BERT in 76 minutes
  1 Apr 2019 · 32 repositories · arXiv:1904.00962
  Syntology community repositories only · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 5 unverified (of 11 harvested samples) · 9 pointer-only (licence)
• Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defenses
  1 Apr 2019 · 1 repository · arXiv:1904.00979
  Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)
• Distilling Task-Specific Knowledge from BERT into Simple Neural Networks
  28 Mar 2019 · 4 repositories · arXiv:1903.12136
  Syntology 9 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 3 honoured, 0 violated, 0 with no contract checked; 6 where Syntology's instrument failed) · 0 unverified (of 9 harvested samples)
• Fine-tune BERT for Extractive Summarization
  25 Mar 2019 · 12 repositories · arXiv:1903.10318
  Syntology community repositories only · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 1 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 1 pointer-only (licence)
• Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence
  22 Mar 2019 · 8 repositories · arXiv:1903.09588
  Syntology official (archive's flag): 8 ran · 19 ran (of which 0 constructed an object rather than computing a result; 15 with no instrument failure: 3 honoured, 0 violated, 12 with no contract checked; 4 where Syntology's instrument failed) · 8 unverified (of 27 harvested samples) · 9 pointer-only (licence)
• Communication-efficient distributed SGD with Sketching
  12 Mar 2019 · 2 repositories · arXiv:1903.04488
  Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
• BERT for Joint Intent Classification and Slot Filling
  28 Feb 2019 · 16 repositories · arXiv:1902.10909
  Syntology 26 ran (of which 0 constructed an object rather than computing a result; 19 with no instrument failure: 1 honoured, 0 violated, 18 with no contract checked; 7 where Syntology's instrument failed) · 6 unverified (of 32 harvested samples) · 5 pointer-only (licence)
• Pretraining-Based Natural Language Generation for Text Summarization
  25 Feb 2019 · 4 repositories · arXiv:1902.09243
  Syntology 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
• BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model
  11 Feb 2019 · 13 repositories · arXiv:1902.04094
  Syntology official (archive's flag): 2 ran · 10 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 2 honoured, 0 violated, 6 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 10 harvested samples) · 2 pointer-only (licence)
• End-to-End Open-Domain Question Answering with BERTserini
  5 Feb 2019 · 1 repository · arXiv:1902.01718
  Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 0 violated, 4 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 8 harvested samples) · 3 pointer-only (licence)
• Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
  4 Feb 2019 · 6 repositories · arXiv:1902.01007
  Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
• A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization
  4 Feb 2019 · 5 repositories · arXiv:1902.01069
  Syntology official (archive's flag): 3 ran · 21 ran (of which 1 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 2 violated, 1 with no contract checked; 16 where Syntology's instrument failed) · 15 unverified (of 36 harvested samples) · 33 pointer-only (licence)
• Parameter-Efficient Transfer Learning for NLP
  2 Feb 2019 · 17 repositories · arXiv:1902.00751
  Syntology official: harvested, nothing ran · 14 ran (of which 3 constructed an object rather than computing a result; 14 with no instrument failure: 0 honoured, 0 violated, 14 with no contract checked; 0 where Syntology's instrument failed) · 8 unverified (of 22 harvested samples) · 3 pointer-only (licence)
• Multi-Task Deep Neural Networks for Natural Language Understanding
  31 Jan 2019 · 7 repositories · arXiv:1901.11504
  Syntology community repositories only · 9 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 1 violated, 4 with no contract checked; 3 where Syntology's instrument failed) · 4 unverified (of 13 harvested samples) · 4 pointer-only (licence)
• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
  25 Jan 2019 · 19 repositories · arXiv:1901.08746
  Syntology official: no sample here; runs from other or unrecorded repositories · 15 ran (of which 0 constructed an object rather than computing a result; 12 with no instrument failure: 1 honoured, 0 violated, 11 with no contract checked; 3 where Syntology's instrument failed) · 10 unverified (of 25 harvested samples) · 3 pointer-only (licence)
• A BERT Baseline for the Natural Questions
  24 Jan 2019 · 3 repositories · arXiv:1901.08634
  Syntology community repositories only · 8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 10 harvested samples)
• TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
  23 Jan 2019 · 23 repositories · arXiv:1901.08149
  Syntology 18 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 1 honoured, 0 violated, 10 with no contract checked; 7 where Syntology's instrument failed) · 8 unverified (of 26 harvested samples) · 7 pointer-only (licence)
• Cross-lingual Language Model Pretraining
  22 Jan 2019 · 17 repositories · arXiv:1901.07291
  Syntology 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 1 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples) · 1 pointer-only (licence)
• Assessing BERT's Syntactic Abilities
  16 Jan 2019 · 3 repositories · arXiv:1901.05287
  Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
• Passage Re-ranking with BERT
  13 Jan 2019 · 6 repositories · arXiv:1901.04085
  Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 13 harvested samples) · 2 pointer-only (licence)
• Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
  9 Jan 2019 · 37 repositories · arXiv:1901.02860
  Syntology official (archive's flag): 8 ran · 67 ran (of which 37 constructed an object rather than computing a result; 49 with no instrument failure: 4 honoured, 1 violated, 44 with no contract checked; 18 where Syntology's instrument failed) · 76 unverified (of 143 harvested samples) · 43 pointer-only (licence)
• Multilingual Constituency Parsing with Self-Attention and Pre-Training
  31 Dec 2018 · 4 repositories · arXiv:1812.11760
  Syntology official (archive's flag): 8 ran · 22 ran (of which 0 constructed an object rather than computing a result; 19 with no instrument failure: 0 honoured, 0 violated, 19 with no contract checked; 3 where Syntology's instrument failed) · 6 unverified (of 28 harvested samples) · 3 pointer-only (licence)
• Conditional BERT Contextual Augmentation
  17 Dec 2018 · 5 repositories · arXiv:1812.06705
  Syntology 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
• Learning RoI Transformer for Detecting Oriented Objects in Aerial Images
  1 Dec 2018 · 1 repository · arXiv:1812.00155
  Syntology 10 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 5 where Syntology's instrument failed) · 1 unverified (of 11 harvested samples) · 11 pointer-only (licence)
• GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism
  16 Nov 2018 · 13 repositories · arXiv:1811.06965
  Syntology 20 ran (of which 0 constructed an object rather than computing a result; 19 with no instrument failure: 0 honoured, 0 violated, 19 with no contract checked; 1 where Syntology's instrument failed) · 5 unverified (of 25 harvested samples) · 16 pointer-only (licence)
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  11 Oct 2018 · 534 repositories · arXiv:1810.04805
  Syntology official: no sample here; runs from other or unrecorded repositories · 300 ran (of which 75 constructed an object rather than computing a result; 235 with no instrument failure: 17 honoured, 4 violated, 214 with no contract checked; 65 where Syntology's instrument failed) · 359 unverified (of 659 harvested samples) · 164 pointer-only (licence)
• Improving the Transformer Translation Model with Document-Level Context
  8 Oct 2018 · 3 repositories · arXiv:1810.03581
  Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 11 unverified (of 12 harvested samples)
• Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks
  1 Oct 2018 · 9 repositories · arXiv:1810.00825
  Syntology official (archive's flag): 3 ran · 8 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 3 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 9 harvested samples) · 4 pointer-only (licence)
• Music Transformer
  12 Sep 2018 · 12 repositories · arXiv:1809.04281
  Syntology 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples) · 2 pointer-only (licence)
• Semi-Autoregressive Neural Machine Translation
  26 Aug 2018 · 1 repository · arXiv:1808.08583
  Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 4 harvested samples)
• Universal Transformers
  10 Jul 2018 · 8 repositories · arXiv:1807.03819
  Syntology community repositories only · 17 ran (of which 8 constructed an object rather than computing a result; 17 with no instrument failure: 1 honoured, 0 violated, 16 with no contract checked; 0 where Syntology's instrument failed) · 8 unverified (of 25 harvested samples) · 24 pointer-only (licence)
• Accelerating Neural Transformer via an Average Attention Network
  2 May 2018 · 1 repository · arXiv:1805.00631
  Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 12 unverified (of 19 harvested samples)
• Adafactor: Adaptive Learning Rates with Sublinear Memory Cost
  11 Apr 2018 · 5 repositories · arXiv:1804.04235
  Syntology 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples)
• Tensor2Tensor for Neural Machine Translation
  16 Mar 2018 · 15 repositories · arXiv:1803.07416
  Syntology official (archive's flag): 1 ran · 30 ran (of which 0 constructed an object rather than computing a result; 29 with no instrument failure: 0 honoured, 0 violated, 29 with no contract checked; 1 where Syntology's instrument failed) · 18 unverified (of 48 harvested samples)
• Self-Attention with Relative Position Representations
  6 Mar 2018 · 13 repositories · arXiv:1803.02155
  Syntology community repositories only · 13 ran (of which 4 constructed an object rather than computing a result; 8 with no instrument failure: 2 honoured, 0 violated, 6 with no contract checked; 5 where Syntology's instrument failed) · 10 unverified (of 23 harvested samples) · 3 pointer-only (licence)
• ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing
  5 Mar 2018 · 2 repositories · arXiv:1803.01837
  Syntology official (archive's flag): 9 ran · 11 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 0 violated, 11 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 13 harvested samples)
• Non-Autoregressive Neural Machine Translation
  7 Nov 2017 · 2 repositories · arXiv:1711.02281
  Syntology official (archive's flag): 1 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 1 pointer-only (licence)
• Weighted Transformer Network for Machine Translation
  6 Nov 2017 · 5 repositories · arXiv:1711.02132
  Syntology 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
• Simple Recurrent Units for Highly Parallelizable Recurrence
  8 Sep 2017 · 11 repositories · arXiv:1709.02755
  Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
• Attention Is All You Need
  12 Jun 2017 · 595 repositories · arXiv:1706.03762
  Syntology official: no sample here; runs from other or unrecorded repositories · 610 ran (of which 293 constructed an object rather than computing a result; 529 with no instrument failure: 45 honoured, 15 violated, 469 with no contract checked; 81 where Syntology's instrument failed) · 336 unverified (of 946 harvested samples) · 451 pointer-only (licence)