Methods › Natural Language Processing › Autoregressive Transformers › Transformer
Transformer
Introduced by Ashish Vaswani et al. in Attention Is All You Need
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
A Transformer is a model architecture that eschews recurrence and instead relies entirely on an attention mechanism to draw global dependencies between input and output. Before Transformers, the dominant sequence transduction models were based on complex recurrent or convolutional neural networks that include an encoder and a decoder. The Transformer also employs an encoder and decoder, but removing recurrence in favor of attention mechanisms allows for significantly more parallelization than methods like RNNs and CNNs.
In Syntology repository known, no verified implementation (declared_code; repositories: 5 listed of 593 in the graph, strongest link: official). This answer is about arXiv:1706.03762, the paper the archive credits with this method.
Code Syntology holds under other names, as the lookup lists them (code_nearby); not a complete list, and none of them is this method.
- Agentic Transformer · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='Agentic Transformer') - Waypoint Transformer · shares_name_tokens (shares name tokens with this name; a different method) · level V2 · retrieve with
get_reference_implementation(name='Waypoint Transformer') - Block-State Transformer · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='Block-State Transformer') - Bootstrapped Transformer · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='Bootstrapped Transformer') - Focused Transformer (FOT) · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='Focused Transformer (FOT)') - Elastic Decision Transformer · shares_name_tokens (shares name tokens with this name; a different method) · level V2 · retrieve with
get_reference_implementation(name='Elastic Decision Transformer')
Origin paper of this answer, arXiv:1706.03762 (its page on this site · its page on Syntology): Syntology's run record for its harvested code, 610 ran (of which 293 constructed an object rather than computing a result; 529 with no instrument failure: 45 honoured, 15 violated, 469 with no contract checked; 81 where Syntology's instrument failed) · 336 unverified (of 946 harvested samples) · 451 pointer-only (licence). The instrument figure counts failures of Syntology's instrument, not of the code. Ran means executed on a synthesized input, not that the code is correct.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
What each level means, quoted from Syntology's server card: V0 “imports”; V1 “executes against a synthesized fixture and returns a value of the recorded type”; V2 “two implementations GENERATED from the paper's signature and spec agree numerically (1e-6) on fixture inputs” “neither of the two ever read the authors' own code, so V2 measures how unambiguously the paper specifies the method, not fidelity to the authors' implementation (measured 2026-09-11, commit 75eaa77: cross_checked_by is a second LLM on 2,628 of 2,831 served samples)”; V3 “paper-derived property tests pass on both of those same two generations.” The card adds: “No rung on this ladder compares the served code against the authors' implementation, so treat every level as evidence about runnability and internal agreement, not about matching the paper's results.” A level quoted in two parts is one definition the card writes as two clauses joined by a double hyphen, each clause quoted on its own (how these levels are quoted).
Papers archive 2025-07-28
archive papers tagged: 13,999 · with a code link: 6,572 · where Syntology ran a sample: 2,248 (1,844 with a run with no instrument failure, 404 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (2,248 of 13,999 tagged: 1,844 with a run with no instrument failure, 404 where every run was a failure of Syntology's instrument)
Page 1 of 140: papers 1 to 100 of 13,999, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
DASViT: Differentiable Architecture Search for Vision Transformer 17 Jul 2025 · 0 repositories · arXiv:2507.13079
-
Best Practices for Large-Scale, Pixel-Wise Crop Mapping and Transfer Learning Workflows 16 Jul 2025 · 1 repository · arXiv:2507.12590
-
DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition 16 Jul 2025 · 1 repository · arXiv:2507.12426
-
Biological Processing Units: Leveraging an Insect Connectome to Pioneer Biofidelic Neural Architectures 15 Jul 2025 · 0 repositories · arXiv:2507.10951
-
Hashed Watermark as a Filter: Defeating Forging and Overwriting Attacks in Weight-based Neural Network Watermarking 15 Jul 2025 · 1 repository · arXiv:2507.11137
-
KV-Latent: Dimensional-level KV Cache Reduction with Frequency-aware Rotary Positional Embedding 15 Jul 2025 · 1 repository · arXiv:2507.11273Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
Langevin Flows for Modeling Neural Latent Dynamics 15 Jul 2025 · 1 repository · arXiv:2507.11531
-
Token Compression Meets Compact Vision Transformers: A Survey and Comparative Evaluation for Edge AI 13 Jul 2025 · 0 repositories · arXiv:2507.09702
-
Learning from Synthetic Labs: Language Models as Auction Participants 12 Jul 2025 · 0 repositories · arXiv:2507.09083
-
Comparative Analysis of Vision Transformers and Traditional Deep Learning Approaches for Automated Pneumonia Detection in Chest X-Rays 11 Jul 2025 · 0 repositories · arXiv:2507.10589
-
A Wireless Foundation Model for Multi-Task Prediction 8 Jul 2025 · 0 repositories · arXiv:2507.05938
-
Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solving 8 Jul 2025 · 1 repository · arXiv:2507.06229Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
Geo-Registration of Terrestrial LiDAR Point Clouds with Satellite Images without GNSS 8 Jul 2025 · 0 repositories · arXiv:2507.05999
-
Growing Transformers: Modular Composition and Layer-wise Expansion on a Frozen Substrate 8 Jul 2025 · 1 repository · arXiv:2507.07129
-
Tile-Based ViT Inference with Visual-Cluster Priors for Zero-Shot Multi-Species Plant Identification 8 Jul 2025 · 1 repository · arXiv:2507.06093
-
AI Generated Text Detection Using Instruction Fine-tuned Large Language and Transformer-Based Models 7 Jul 2025 · 0 repositories · arXiv:2507.05157
-
Emergent Semantics Beyond Token Embeddings: Transformer LMs with Frozen Visual Unicode Representations 7 Jul 2025 · 1 repository · arXiv:2507.04886
-
Estimating Interventional Distributions with Uncertain Causal Graphs through Meta-Learning 7 Jul 2025 · 0 repositories · arXiv:2507.05526
-
SV-DRR: High-Fidelity Novel View X-Ray Synthesis Using Diffusion Model 7 Jul 2025 · 1 repository · arXiv:2507.05148
-
DeepGesture: A conversational gesture synthesis system based on emotions and semantics 3 Jul 2025 · 1 repository · arXiv:2507.03147
-
Fast and Simplex: 2-Simplicial Attention in Triton 3 Jul 2025 · 0 repositories · arXiv:2507.02754
-
AC-DiT: Adaptive Coordination Diffusion Transformer for Mobile Manipulation 2 Jul 2025 · 0 repositories · arXiv:2507.01961
-
Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer 2 Jul 2025 · 1 repository · arXiv:2507.02199Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 2 violated, 4 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 11 harvested samples) · 11 pointer-only (licence)
-
A Unified Transformer-Based Framework with Pretraining For Whole Body Grasping Motion Generation 1 Jul 2025 · 1 repository · arXiv:2507.00676
-
Large Language Models Don't Make Sense of Word Problems. A Scoping Review from a Mathematics Education Perspective 30 Jun 2025 · 0 repositories · arXiv:2506.24006
-
Mamba-FETrack V2: Revisiting State Space Model for Frame-Event based Visual Object Tracking 30 Jun 2025 · 1 repository · arXiv:2506.23783
-
CycleVAR: Repurposing Autoregressive Model for Unsupervised One-Step Image Translation 29 Jun 2025 · 1 repository · arXiv:2506.23347
-
Attention to Burstiness: Low-Rank Bilinear Prompt Tuning 28 Jun 2025 · 1 repository · arXiv:2506.22908
-
Boosting Generative Adversarial Transferability with Self-supervised Vision Transformer Features 26 Jun 2025 · 1 repository · arXiv:2506.21046
-
Chain-of-Thought Enhanced Shallow Transformers for Wireless Symbol Detection 26 Jun 2025 · 0 repositories · arXiv:2506.21093
-
DBConformer: Dual-Branch Convolutional Transformer for EEG Decoding 26 Jun 2025 · 1 repository · arXiv:2506.21140
-
Early Stopping Tabular In-Context Learning 26 Jun 2025 · 0 repositories · arXiv:2506.21387
-
Forecasting Geopolitical Events with a Sparse Temporal Fusion Transformer and Gaussian Process Hybrid: A Case Study in Middle Eastern and U.S. Conflict Dynamics 26 Jun 2025 · 0 repositories · arXiv:2506.20935
-
M2SFormer: Multi-Spectral and Multi-Scale Attention with Edge-Aware Difficulty Guidance for Image Forgery Localization 26 Jun 2025 · 0 repositories · arXiv:2506.20922
-
Progtuning: Progressive Fine-tuning Framework for Transformer-based Language Models 26 Jun 2025 · 0 repositories · arXiv:2506.21119
-
Real-Time ESFP: Estimating, Smoothing, Filtering, and Pose-Mapping 26 Jun 2025 · 1 repository · arXiv:2506.21234
-
Transformer-Based Spatial-Temporal Counterfactual Outcomes Estimation 26 Jun 2025 · 1 repository · arXiv:2506.21154Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
Video Virtual Try-on with Conditional Diffusion Transformer Inpainter 26 Jun 2025 · 0 repositories · arXiv:2506.21270
-
A foundation model with multi-variate parallel attention to generate neuronal activity 25 Jun 2025 · 1 repository · arXiv:2506.20354
-
AdvMIM: Adversarial Masked Image Modeling for Semi-Supervised Medical Image Segmentation 25 Jun 2025 · 0 repositories · arXiv:2506.20563
-
Differential Transformer-driven 6G Physical Layer for Collaborative Perception Enhancement 25 Jun 2025 · 0 repositories · arXiv:2506.20597
-
Feature Hallucination for Self-supervised Action Recognition 25 Jun 2025 · 0 repositories · arXiv:2506.20342
-
ITFormer: Bridging Time Series and Natural Language for Multi-Modal QA with Large-Scale Multitask Dataset 25 Jun 2025 · 0 repositories · arXiv:2506.20093
-
Med-Art: Diffusion Transformer for 2D Medical Text-to-Image Generation 25 Jun 2025 · 0 repositories · arXiv:2506.20449
-
MS-IQA: A Multi-Scale Feature Fusion Network for PET/CT Image Quality Assessment 25 Jun 2025 · 1 repository · arXiv:2506.20200
-
Probing AI Safety with Source Code 25 Jun 2025 · 1 repository · arXiv:2506.20471
-
X-SiT: Inherently Interpretable Surface Vision Transformers for Dementia Diagnosis 25 Jun 2025 · 0 repositories · arXiv:2506.20267
-
Lost in Translation? Converting RegExes for Log Parsing into Dynatrace Pattern Language 24 Jun 2025 · 0 repositories · arXiv:2506.19539
-
Toward the Explainability of Protein Language Models for Sequence Design 24 Jun 2025 · 0 repositories · arXiv:2506.19532
-
Vision Transformer-Based Time-Series Image Reconstruction for Cloud-Filling Applications 24 Jun 2025 · 0 repositories · arXiv:2506.19591
-
WebGuard++:Interpretable Malicious URL Detection via Bidirectional Fusion of HTML Subgraphs and Multi-Scale Convolutional BERT 24 Jun 2025 · 0 repositories · arXiv:2506.19356
-
A Study of Dynamic Stock Relationship Modeling and S&P500 Price Forecasting Based on Differential Graph Transformer 23 Jun 2025 · 0 repositories · arXiv:2506.18717
-
Deep CNN Face Matchers Inherently Support Revocable Biometric Templates 23 Jun 2025 · 0 repositories · arXiv:2506.18731
-
Fully Few-shot Class-incremental Audio Classification Using Multi-level Embedding Extractor and Ridge Regression Classifier 23 Jun 2025 · 1 repository · arXiv:2506.18406
-
MuseControlLite: Multifunctional Music Generation with Lightweight Conditioners 23 Jun 2025 · 0 repositories · arXiv:2506.18729
-
Rethinking Decoder Design: Improving Biomarker Segmentation Using Depth-to-Space Restoration and Residual Linear Attention 23 Jun 2025 · 1 repository · arXiv:2506.18335Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
Security Assessment of DeepSeek and GPT Series Models against Jailbreak Attacks 23 Jun 2025 · 0 repositories · arXiv:2506.18543
-
SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applications 23 Jun 2025 · 0 repositories · arXiv:2506.18951
-
Transformer World Model for Sample Efficient Multi-Agent Reinforcement Learning 23 Jun 2025 · 1 repository · arXiv:2506.18537
-
Leveraging Large Language Model for Intelligent Log Processing and Autonomous Debugging in Cloud AI Platforms 22 Jun 2025 · 0 repositories · arXiv:2506.17900
-
In-Context Learning Strategies Emerge Rationally 21 Jun 2025 · 0 repositories · arXiv:2506.17859
-
MTSIC: Multi-stage Transformer-based GAN for Spectral Infrared Image Colorization 21 Jun 2025 · 0 repositories · arXiv:2506.17540
-
IsoNet: Causal Analysis of Multimodal Transformers for Neuromuscular Gesture Classification 20 Jun 2025 · 0 repositories · arXiv:2506.16744
-
Leveraging LLMs to Assess Tutor Moves in Real-Life Dialogues: A Feasibility Study 20 Jun 2025 · 0 repositories · arXiv:2506.17410
-
Optimal Depth of Neural Networks 20 Jun 2025 · 0 repositories · arXiv:2506.16862
-
Trans²-CBCT: A Dual-Transformer Framework for Sparse-View CBCT Reconstruction 20 Jun 2025 · 0 repositories · arXiv:2506.17425
-
UniFork: Exploring Modality Alignment for Unified Multimodal Understanding and Generation 20 Jun 2025 · 1 repository · arXiv:2506.17202Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 8 where Syntology's instrument failed) · 0 unverified (of 8 harvested samples) · 8 pointer-only (licence)
-
VMRA-MaR: An Asymmetry-Aware Temporal Framework for Longitudinal Breast Cancer Risk Prediction 20 Jun 2025 · 1 repository · arXiv:2506.17412
-
CapsDT: Diffusion-Transformer for Capsule Robot Manipulation 19 Jun 2025 · 0 repositories · arXiv:2506.16263
-
Do We Talk to Robots Like Therapists, and Do They Respond Accordingly? Language Alignment in AI Emotional Support 19 Jun 2025 · 0 repositories · arXiv:2506.16473
-
Exoplanet Classification through Vision Transformers with Temporal Image Analysis 19 Jun 2025 · 0 repositories · arXiv:2506.16597
-
STAR-Pose: Efficient Low-Resolution Video Human Pose Estimation via Spatial-Temporal Adaptive Super-Resolution 19 Jun 2025 · 0 repositories · arXiv:2506.16061
-
DiscoSG: Towards Discourse-Level Text Scene Graph Parsing through Iterative Graph Refinement 18 Jun 2025 · 2 repositories · arXiv:2506.15583Syntology official (archive's flag): 17 ran · 17 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 1 honoured, 1 violated, 7 with no contract checked; 8 where Syntology's instrument failed) · 2 unverified (of 19 harvested samples) · 19 pointer-only (licence)
-
I Know Which LLM Wrote Your Code Last Summer: LLM generated Code Stylometry for Authorship Attribution 18 Jun 2025 · 0 repositories · arXiv:2506.17323
-
In-Context Learning for Gradient-Free Receiver Adaptation: Principles, Applications, and Theory 18 Jun 2025 · 0 repositories · arXiv:2506.15176
-
Advances in Compliance Detection: Novel Models Using Vision-Based Tactile Sensors 17 Jun 2025 · 0 repositories · arXiv:2506.14980
-
ImpliRet: Benchmarking the Implicit Fact Retrieval Challenge 17 Jun 2025 · 1 repository · arXiv:2506.14407Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
PoseGRAF: Geometric-Reinforced Adaptive Fusion for Monocular 3D Human Pose Estimation 17 Jun 2025 · 1 repository · arXiv:2506.14596
-
Scaling Intelligence: Designing Data Centers for Next-Gen Language Models 17 Jun 2025 · 0 repositories · arXiv:2506.15006
-
Toward a Graph Foundation Model: Pre-Training Transformers With Random Walks 17 Jun 2025 · 0 repositories · arXiv:2506.14098
-
A Gravity-informed Spatiotemporal Transformer for Human Activity Intensity Prediction 16 Jun 2025 · 0 repositories · arXiv:2506.13678
-
FOAM: A General Frequency-Optimized Anti-Overlapping Framework for Overlapping Object Perception 16 Jun 2025 · 0 repositories · arXiv:2506.13501
-
HierVL: Semi-Supervised Segmentation leveraging Hierarchical Vision-Language Synergy with Dynamic Text-Spatial Query Alignment 16 Jun 2025 · 0 repositories · arXiv:2506.13925
-
MambaMia: A State-Space-Model-Based Compression for Efficient Video Understanding in Large Multimodal Models 16 Jun 2025 · 0 repositories · arXiv:2506.13564
-
MT-PCR: A Hybrid Mamba-Transformer with Spatial Serialization for Hierarchical Point Cloud Registration 16 Jun 2025 · 0 repositories · arXiv:2506.13183
-
Overcoming Occlusions in the Wild: A Multi-Task Age Head Approach to Age Estimation 16 Jun 2025 · 0 repositories · arXiv:2506.13445
-
PF-LHM: 3D Animatable Avatar Reconstruction from Pose-free Articulated Human Images 16 Jun 2025 · 0 repositories · arXiv:2506.13766
-
Alphabet Index Mapping: Jailbreaking LLMs through Semantic Dissimilarity 15 Jun 2025 · 0 repositories · arXiv:2506.12685
-
Cross-architecture universal feature coding via distribution alignment 15 Jun 2025 · 0 repositories · arXiv:2506.12737
-
Evaluating Cell Type Inference in Vision Language Models Under Varying Visual Context 15 Jun 2025 · 1 repository · arXiv:2506.12683
-
Frequency Dynamic Convolutions for Sound Event Detection 15 Jun 2025 · 0 repositories · arXiv:2506.12785
-
GM-LDM: Latent Diffusion Model for Brain Biomarker Identification through Functional Data-Driven Gray Matter Synthesis 15 Jun 2025 · 0 repositories · arXiv:2506.12719
-
BSA: Ball Sparse Attention for Large-scale Geometries 14 Jun 2025 · 1 repository · arXiv:2506.12541Syntology official (archive's flag): 2 ran · 2 ran (of which 1 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 1 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)
-
Is your batch size the problem? Revisiting the Adam-SGD gap in language modeling 14 Jun 2025 · 0 repositories · arXiv:2506.12543
-
Language Models Enable Data-Augmented Synthesis Planning for Inorganic Materials 14 Jun 2025 · 0 repositories · arXiv:2506.12557
-
Parkinson's Disease Freezing of Gait (FoG) Symptom Detection Using Machine Learning from Wearable Sensor Data 14 Jun 2025 · 0 repositories · arXiv:2506.12561
-
Structural feature enhanced transformer for fine-grained image recognition 14 Jun 2025 · 0 repositories
-
Bubble Dynamics Transformer: Microrheology at Ultra-High Strain Rates 13 Jun 2025 · 0 repositories · arXiv:2506.11936
-
GraphGSOcc: Semantic-Geometric Graph Transformer with Dynamic-Static Decoupling for 3D Gaussian Splatting-based Occupancy Prediction 13 Jun 2025 · 0 repositories · arXiv:2506.14825
-
Identifying Helpful Context for LLM-based Vulnerability Repair: A Preliminary Study 13 Jun 2025 · 0 repositories · arXiv:2506.11561
Tasks archive 2025-07-28
20 shown of 2,170 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
| Task | Papers |
|---|---|
| Language Modelling | 1,237 |
| Decoder | 1,068 |
| Language Modeling | 949 |
| Translation | 758 |
| Machine Translation | 718 |
| Semantic Segmentation | 679 |
| Object Detection | 552 |
| Image Classification | 524 |
| Question Answering | 514 |
| object-detection | 497 |
| Retrieval | 455 |
| Sentence | 454 |
| Segmentation | 448 |
| Representation Learning | 428 |
| image-classification | 414 |
| Large Language Model | 409 |
| Time Series | 377 |
| Object | 352 |
| Text Generation | 317 |
| Transfer Learning | 294 |
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections