{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/itransformer-inverted-transformers-are","title":"iTransformer: Inverted Transformers Are Effective for Time Series Forecasting","arxiv_id":"2310.06625","date":"2023-10-10","proceeding":null,"authors":["Yong liu","Tengge Hu","Haoran Zhang","Haixu Wu","Shiyu Wang","Lintao Ma","Mingsheng Long"],"abstract":"The recent boom of linear forecasting models questions the ongoing passion for architectural modifications of Transformer-based forecasters. These forecasters leverage Transformers to model the global dependencies over temporal tokens of time series, with each token formed by multiple variates of the same timestamp. However, Transformers are challenged in forecasting series with larger lookback windows due to performance degradation and computation explosion. Besides, the embedding for each temporal token fuses multiple variates that represent potential delayed events and distinct physical measurements, which may fail in learning variate-centric representations and result in meaningless attention maps. In this work, we reflect on the competent duties of Transformer components and repurpose the Transformer architecture without any modification to the basic components. We propose iTransformer that simply applies the attention and feed-forward network on the inverted dimensions. Specifically, the time points of individual series are embedded into variate tokens which are utilized by the attention mechanism to capture multivariate correlations; meanwhile, the feed-forward network is applied for each variate token to learn nonlinear representations. The iTransformer model achieves state-of-the-art on challenging real-world datasets, which further empowers the Transformer family with promoted performance, generalization ability across different variates, and better utilization of arbitrary lookback windows, making it a nice alternative as the fundamental backbone of time series forecasting. Code is available at this repository: https://github.com/thuml/iTransformer.","url_abs":"https://arxiv.org/abs/2310.06625v4","url_pdf":"https://arxiv.org/pdf/2310.06625v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"itransformer-inverted-transformers-are","repo_url":"https://github.com/thuml/iTransformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"itransformer-inverted-transformers-are","repo_url":"https://github.com/Hannibal046/GridTST","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"itransformer-inverted-transformers-are","repo_url":"https://github.com/hughxx/tsf-new-paper-taste","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"itransformer-inverted-transformers-are","repo_url":"https://github.com/kwuking/TimeMixer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"itransformer-inverted-transformers-are","repo_url":"https://github.com/lss-1138/SegRNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"itransformer-inverted-transformers-are","repo_url":"https://github.com/lucidrains/iTransformer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"itransformer-inverted-transformers-are","repo_url":"https://github.com/master-plc/fredf","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"itransformer-inverted-transformers-are","repo_url":"https://github.com/sanjaylopa22/QCAAPatchTF","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"itransformer-inverted-transformers-are","repo_url":"https://github.com/taohan10200/weather-5k","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"itransformer-inverted-transformers-are","repo_url":"https://github.com/thuml/Time-Series-Library","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"itransformer-inverted-transformers-are","repo_url":"https://github.com/WenjieDu/PyPOTS","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series-forecasting","task_name":"Time Series Forecasting"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/time-series-forecasting-on-etth1-336-1","task":"Time Series Forecasting","dataset":"ETTh1 (336) Multivariate","model":"iTransformer","rank_in_archive_order":59,"of":72,"metrics":{"MAE":"0.458","MSE":"0.487"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.06625","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.06625"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thuml/Time-Series-Library","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kwuking/TimeMixer","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thuml/iTransformer","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hughxx/tsf-new-paper-taste","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sanjaylopa22/QCAAPatchTF","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Hannibal046/GridTST","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/taohan10200/weather-5k","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lss-1138/SegRNN","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/master-plc/fredf","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/WenjieDu/PyPOTS","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lucidrains/iTransformer","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":2,"ran_violates":4,"ran":1,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"listed":{"samples":10,"ran":8,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7f1040f5e3991d5e","entry":"identity","repo":"lucidrains/iTransformer","repo_kind":"listed","path":"iTransformer/iTransformer.py","file_url":"https://github.com/lucidrains/iTransformer/blob/HEAD/iTransformer/iTransformer.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":2,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f1040f5e3991d5e"}},{"code_sha256_prefix":"16b9b78d429636b7","entry":"PositionalEncoding","repo":"lss-1138/SegRNN","repo_kind":"listed","path":"layers/PatchTST_layers.py","file_url":"https://github.com/lss-1138/SegRNN/blob/HEAD/layers/PatchTST_layers.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"16b9b78d429636b7"}},{"code_sha256_prefix":"60fff7c3c400d7ff","entry":"default","repo":"lucidrains/iTransformer","repo_kind":"listed","path":"iTransformer/attend.py","file_url":"https://github.com/lucidrains/iTransformer/blob/HEAD/iTransformer/attend.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"60fff7c3c400d7ff"}},{"code_sha256_prefix":"99b3563ebc6734ad","entry":"default","repo":"lucidrains/iTransformer","repo_kind":"listed","path":"iTransformer/iTransformer.py","file_url":"https://github.com/lucidrains/iTransformer/blob/HEAD/iTransformer/iTransformer.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"99b3563ebc6734ad"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"lucidrains/iTransformer","repo_kind":"listed","path":"iTransformer/attend.py","file_url":"https://github.com/lucidrains/iTransformer/blob/HEAD/iTransformer/attend.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"b7c3487f192e31b7","entry":"exists","repo":"lucidrains/iTransformer","repo_kind":"listed","path":"iTransformer/iTransformer.py","file_url":"https://github.com/lucidrains/iTransformer/blob/HEAD/iTransformer/iTransformer.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b7c3487f192e31b7"}},{"code_sha256_prefix":"72b9272542f279c6","entry":"get_activation_fn","repo":"lss-1138/SegRNN","repo_kind":"listed","path":"layers/PatchTST_layers.py","file_url":"https://github.com/lss-1138/SegRNN/blob/HEAD/layers/PatchTST_layers.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"72b9272542f279c6"}},{"code_sha256_prefix":"52e10a7d6b0fa036","entry":"once","repo":"lucidrains/iTransformer","repo_kind":"listed","path":"iTransformer/attend.py","file_url":"https://github.com/lucidrains/iTransformer/blob/HEAD/iTransformer/attend.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"52e10a7d6b0fa036"}},{"code_sha256_prefix":"c3fc6992d1e467f3","entry":"Coord2dPosEncoding","repo":"lss-1138/SegRNN","repo_kind":"listed","path":"layers/PatchTST_layers.py","file_url":"https://github.com/lss-1138/SegRNN/blob/HEAD/layers/PatchTST_layers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c3fc6992d1e467f3"}},{"code_sha256_prefix":"6182326691dd827b","entry":"pack_one","repo":"lucidrains/iTransformer","repo_kind":"listed","path":"iTransformer/iTransformer2D.py","file_url":"https://github.com/lucidrains/iTransformer/blob/HEAD/iTransformer/iTransformer2D.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6182326691dd827b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}