Papers › Transformers in Time Series: A Survey

Transformers in Time Series: A Survey

15 Feb 2022arXiv:2202.07125archive 2025-07-28

Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, Liang Sun

Transformers have achieved superior performances in many tasks in natural language processing and computer vision, which also triggered great interest in the time series community. Among multiple advantages of Transformers, the ability to capture long-range dependencies and interactions is especially attractive for time series modeling, leading to exciting progress in various time series applications. In this paper, we systematically review Transformer schemes for time series modeling by highlighting their strengths as well as limitations. In particular, we examine the development of time series Transformers in two perspectives. From the perspective of network structure, we summarize the adaptations and modifications that have been made to Transformers in order to accommodate the challenges in time series analysis. From the perspective of applications, we categorize time series Transformers based on common tasks including forecasting, anomaly detection, and classification. Empirically, we perform robust analysis, model size analysis, and seasonal-trend decomposition analysis to study how Transformers perform in time series. Finally, we discuss and suggest future directions to provide useful research guidance. To the best of our knowledge, this paper is the first work to comprehensively and systematically summarize the recent advances of Transformers for modeling time series data. We hope this survey will ignite further research interests in time series Transformers.

PaperPDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

qingsongedu/time-series-transformers-review officialmentioned in papermentioned on GitHubMITSyntology: not harvested report
MAZiqing/FEDformer mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to SEMPO: Lightweight Foundation Models for Time Series Forecasting (arXiv:2510.19710; that paper's own run record: 16 ran (of which 7 constructed an object rather than computing a result; 12 with no instrument failure: 4 honoured, 0 violated, 8 with no contract checked; 4 where Syntology's instrument failed) · 7 unverified); nothing checks that they implement this paper's method. report
chengyui/nuwats mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to NuwaTS: a Foundation Model Mending Every Incomplete Time Series (arXiv:2405.15317; that paper's own run record: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
damo-di-ml/icml2022-fedformer mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting (arXiv:2201.12740; that paper's own run record: 14 ran (of which 8 constructed an object rather than computing a result; 11 with no instrument failure: 1 honoured, 0 violated, 10 with no contract checked; 3 where Syntology's instrument failed) · 4 unverified); nothing checks that they implement this paper's method. report
damo-di-ml/neurips2023-one-fits-all mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to One Fits All:Power General Time Series Analysis by Pretrained LM (arXiv:2302.11939; that paper's own run record: 6 ran (of which 6 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified; every one of the 6 samples that ran constructed an object rather than computing a result) and From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting... (arXiv:2409.17515; that paper's own run record: 11 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 8 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
damo-di-ml/one_fits_all mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to One Fits All:Power General Time Series Analysis by Pretrained LM (arXiv:2302.11939; that paper's own run record: 6 ran (of which 6 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified; every one of the 6 samples that ran constructed an object rather than computing a result) and SEMPO: Lightweight Foundation Models for Time Series Forecasting (arXiv:2510.19710; that paper's own run record: 16 ran (of which 7 constructed an object rather than computing a result; 12 with no instrument failure: 4 honoured, 0 violated, 8 with no contract checked; 4 where Syntology's instrument failed) · 7 unverified); nothing checks that they implement this paper's method. report
decisionintelligence/pathformer mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Pathformer: Multi-scale Transformers with Adaptive Pathways for Time Series Forecasting (arXiv:2402.05956; that paper's own run record: 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 5 unverified); nothing checks that they implement this paper's method. report
haochenglouis/robusttsf mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to RobustTSF: Towards Theory and Design of Robust Time Series Forecasting with Anomalies (arXiv:2402.02032; that paper's own run record: 10 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 10 where Syntology's instrument failed) · 2 unverified); nothing checks that they implement this paper's method. report
kimmeen/time-llm mentioned on GitHubpytorchApache-2.0Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (arXiv:2310.01728; that paper's own run record: 7 ran (of which 7 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified; every one of the 7 samples that ran constructed an object rather than computing a result); nothing checks that they implement this paper's method. report
wxie9/card mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to CARD: Channel Aligned Robust Blend Transformer for Time Series Forecasting (arXiv:2305.12095; that paper's own run record: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
zichuan-liu/contralsp mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Explaining Time Series via Contrastive and Locally Sparse Perturbations (arXiv:2401.08552; that paper's own run record: 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Anomaly DetectionSurveyTime SeriesTime Series Analysis

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

Absolute Position EncodingsAdamAttentionBPEDense ConnectionsDropoutLabel SmoothingLayer NormalizationLinear LayerMulti-Head AttentionPosition-Wise Feed-Forward LayerResidual ConnectionSoftmaxTransformer

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections