{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/prformer-pyramidal-recurrent-transformer-for","title":"PRformer: Pyramidal Recurrent Transformer for Multivariate Time Series Forecasting","arxiv_id":"2408.10483","date":"2024-08-20","proceeding":null,"authors":["Yongbo Yu","Weizhong Yu","Feiping Nie","Xuelong Li"],"abstract":"The self-attention mechanism in Transformer architecture, invariant to sequence order, necessitates positional embeddings to encode temporal order in time series prediction. We argue that this reliance on positional embeddings restricts the Transformer's ability to effectively represent temporal sequences, particularly when employing longer lookback windows. To address this, we introduce an innovative approach that combines Pyramid RNN embeddings(PRE) for univariate time series with the Transformer's capability to model multivariate dependencies. PRE, utilizing pyramidal one-dimensional convolutional layers, constructs multiscale convolutional features that preserve temporal order. Additionally, RNNs, layered atop these features, learn multiscale time series representations sensitive to sequence order. This integration into Transformer models with attention mechanisms results in significant performance enhancements. We present the PRformer, a model integrating PRE with a standard Transformer encoder, demonstrating state-of-the-art performance on various real-world datasets. This performance highlights the effectiveness of our approach in leveraging longer lookback windows and underscores the critical role of robust temporal representations in maximizing Transformer's potential for prediction tasks. Code is available at this repository: \\url{https://github.com/usualheart/PRformer}.","url_abs":"https://arxiv.org/abs/2408.10483v1","url_pdf":"https://arxiv.org/pdf/2408.10483v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"prformer-pyramidal-recurrent-transformer-for","repo_url":"https://github.com/usualheart/prformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"multivariate-time-series-forecasting","task_name":"Multivariate Time Series Forecasting"},{"task_slug":"temporal-sequences","task_name":"Temporal Sequences"},{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series-forecasting","task_name":"Time Series Forecasting"},{"task_slug":"time-series-prediction","task_name":"Time Series Prediction"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/multivariate-time-series-forecasting-on-etth1-1","task":"Multivariate Time Series Forecasting","dataset":"ETTh1 (192) Multivariate","model":"PRformer","rank_in_archive_order":2,"of":3,"metrics":{"MAE":"0.410","MSE":"0.397"},"uses_additional_data":false},{"leaderboard":"/sota/multivariate-time-series-forecasting-on-etth1","task":"Multivariate Time Series Forecasting","dataset":"ETTh1 (96) Multivariate","model":"PRformer","rank_in_archive_order":2,"of":3,"metrics":{"MAE":"0.383","MSE":"0.354"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth1-192-1","task":"Time Series Forecasting","dataset":"ETTh1 (192) Multivariate","model":"PRformer","rank_in_archive_order":6,"of":17,"metrics":{"MSE":"0.397"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth1-336-1","task":"Time Series Forecasting","dataset":"ETTh1 (336) Multivariate","model":"PRformer","rank_in_archive_order":26,"of":72,"metrics":{"MSE":"0.427"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth1-720-1","task":"Time Series Forecasting","dataset":"ETTh1 (720) Multivariate","model":"PRformer","rank_in_archive_order":16,"of":22,"metrics":{"MSE":"0.489"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth1-96-1","task":"Time Series Forecasting","dataset":"ETTh1 (96) Multivariate","model":"PRformer","rank_in_archive_order":4,"of":15,"metrics":{"MSE":"0.354"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth2-192-1","task":"Time Series Forecasting","dataset":"ETTh2 (192) Multivariate","model":"PRformer","rank_in_archive_order":9,"of":16,"metrics":{"MSE":"0.332"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth2-336-1","task":"Time Series Forecasting","dataset":"ETTh2 (336) Multivariate","model":"PRformer","rank_in_archive_order":11,"of":20,"metrics":{"MSE":"0.361"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth2-720-1","task":"Time Series Forecasting","dataset":"ETTh2 (720) Multivariate","model":"PRformer","rank_in_archive_order":9,"of":20,"metrics":{"MSE":"0.396"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth2-96-1","task":"Time Series Forecasting","dataset":"ETTh2 (96) Multivariate","model":"PRformer","rank_in_archive_order":6,"of":16,"metrics":{"MSE":"0.268"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-ettm1-192-1","task":"Time Series Forecasting","dataset":"ETTm1 (192) Multivariate","model":"PRformer","rank_in_archive_order":3,"of":9,"metrics":{"MSE":"0.324"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-ettm1-336-1","task":"Time Series Forecasting","dataset":"ETTm1 (336) Multivariate","model":"PRformer","rank_in_archive_order":3,"of":8,"metrics":{"MSE":"0.362"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-ettm1-720-1","task":"Time Series Forecasting","dataset":"ETTm1 (720) Multivariate","model":"PRformer","rank_in_archive_order":6,"of":8,"metrics":{"MSE":"0.426"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-ettm1-96-1","task":"Time Series Forecasting","dataset":"ETTm1 (96) Multivariate","model":"PRformer","rank_in_archive_order":3,"of":9,"metrics":{"MSE":"0.278"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-ettm2-192-1","task":"Time Series Forecasting","dataset":"ETTm2 (192) Multivariate","model":"PRformer","rank_in_archive_order":5,"of":9,"metrics":{"MSE":"0.219"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-ettm2-336-1","task":"Time Series Forecasting","dataset":"ETTm2 (336) Multivariate","model":"PRformer","rank_in_archive_order":4,"of":8,"metrics":{"MSE":"0.272"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-ettm2-720-1","task":"Time Series Forecasting","dataset":"ETTm2 (720) Multivariate","model":"PRformer","rank_in_archive_order":5,"of":8,"metrics":{"MSE":"0.359"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-ettm2-96-1","task":"Time Series Forecasting","dataset":"ETTm2 (96) Multivariate","model":"PRformer","rank_in_archive_order":3,"of":9,"metrics":{"MSE":"0.162"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-electricity-192","task":"Time Series Forecasting","dataset":"Electricity (192)","model":"PRformer","rank_in_archive_order":5,"of":8,"metrics":{"MSE":"0.148"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-electricity-336","task":"Time Series Forecasting","dataset":"Electricity (336)","model":"PRformer","rank_in_archive_order":3,"of":9,"metrics":{"MSE":"0.161"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-electricity-720","task":"Time Series Forecasting","dataset":"Electricity (720)","model":"PRformer","rank_in_archive_order":3,"of":8,"metrics":{"MSE":"0.185"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-electricity-96","task":"Time Series Forecasting","dataset":"Electricity (96)","model":"PRformer","rank_in_archive_order":3,"of":11,"metrics":{"MSE":"0.127"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-traffic-192","task":"Time Series Forecasting","dataset":"Traffic (192)","model":"PRformer","rank_in_archive_order":1,"of":3,"metrics":{"MSE":"0.372"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-traffic-336","task":"Time Series Forecasting","dataset":"Traffic (336)","model":"PRformer","rank_in_archive_order":2,"of":3,"metrics":{"MSE":"0.385"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-traffic-720","task":"Time Series Forecasting","dataset":"Traffic (720)","model":"PRformer","rank_in_archive_order":1,"of":3,"metrics":{"MSE":"0.421"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-traffic-96","task":"Time Series Forecasting","dataset":"Traffic (96)","model":"PRformer","rank_in_archive_order":1,"of":3,"metrics":{"MSE":"0.353"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather-192","task":"Time Series Forecasting","dataset":"Weather (192)","model":"PRformer","rank_in_archive_order":4,"of":13,"metrics":{"MSE":"0.188"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather-336","task":"Time Series Forecasting","dataset":"Weather (336)","model":"PRformer","rank_in_archive_order":6,"of":11,"metrics":{"MSE":"0.241"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather-720","task":"Time Series Forecasting","dataset":"Weather (720)","model":"PRformer","rank_in_archive_order":9,"of":11,"metrics":{"MSE":"0.326"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather-96","task":"Time Series Forecasting","dataset":"Weather (96)","model":"PRformer","rank_in_archive_order":4,"of":12,"metrics":{"MSE":"0.144"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2408.10483","atlas_url":"https://app.syntology.ai/?focus=2408.10483","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.10483"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/usualheart/prformer","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":2},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"21c9eb7b89621d6a","entry":"countResultWindows","repo":"usualheart/prformer","repo_kind":"official","path":"layers/WindowListSplit.py","file_url":"https://github.com/usualheart/prformer/blob/HEAD/layers/WindowListSplit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"21c9eb7b89621d6a"}},{"code_sha256_prefix":"e0b14c57965ffd22","entry":"splitWindowList","repo":"usualheart/prformer","repo_kind":"official","path":"layers/WindowListSplit.py","file_url":"https://github.com/usualheart/prformer/blob/HEAD/layers/WindowListSplit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e0b14c57965ffd22"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}