{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generative-pretrained-hierarchical","title":"Generative Pretrained Hierarchical Transformer for Time Series Forecasting","arxiv_id":"2402.16516","date":"2024-02-26","proceeding":null,"authors":["Zhiding Liu","Jiqian Yang","Mingyue Cheng","Yucong Luo","Zhi Li"],"abstract":"Recent efforts have been dedicated to enhancing time series forecasting accuracy by introducing advanced network architectures and self-supervised pretraining strategies. Nevertheless, existing approaches still exhibit two critical drawbacks. Firstly, these methods often rely on a single dataset for training, limiting the model's generalizability due to the restricted scale of the training data. Secondly, the one-step generation schema is widely followed, which necessitates a customized forecasting head and overlooks the temporal dependencies in the output series, and also leads to increased training costs under different horizon length settings. To address these issues, we propose a novel generative pretrained hierarchical transformer architecture for forecasting, named \\textbf{GPHT}. There are two aspects of key designs in GPHT. On the one hand, we advocate for constructing a mixed dataset under the channel-independent assumption for pretraining our model, comprising various datasets from diverse data scenarios. This approach significantly expands the scale of training data, allowing our model to uncover commonalities in time series data and facilitating improved transfer to specific datasets. On the other hand, GPHT employs an auto-regressive forecasting approach, effectively modeling temporal dependencies in the output series. Importantly, no customized forecasting head is required, enabling \\textit{a single model to forecast at arbitrary horizon settings.} We conduct sufficient experiments on eight datasets with mainstream self-supervised pretraining models and supervised models. The results demonstrated that GPHT surpasses the baseline models across various fine-tuning and zero/few-shot learning settings in the traditional long-term forecasting task. We make our codes publicly available\\footnote{https://github.com/icantnamemyself/GPHT}.","url_abs":"https://arxiv.org/abs/2402.16516v2","url_pdf":"https://arxiv.org/pdf/2402.16516v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generative-pretrained-hierarchical","repo_url":"https://github.com/icantnamemyself/gpht","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"generative-pretrained-hierarchical","repo_url":"https://github.com/mingyue-cheng/crosstimenet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series-forecasting","task_name":"Time Series Forecasting"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/time-series-forecasting-on-etth1-336-1","task":"Time Series Forecasting","dataset":"ETTh1 (336) Multivariate","model":"GPHT","rank_in_archive_order":29,"of":72,"metrics":{"MAE":"0.423","MSE":"0.430"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth1-336-1","task":"Time Series Forecasting","dataset":"ETTh1 (336) Multivariate","model":"GPHT*","rank_in_archive_order":45,"of":72,"metrics":{"MAE":"0.432","MSE":"0.456"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.16516","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.16516"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/icantnamemyself/gpht","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mingyue-cheng/crosstimenet","reach":{"status":"ok"}}],"summary":{"ran":1,"ran_honours":1,"ran_draft_wrong":1,"ran_fixture":2,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2af0df7394e58e06","entry":"conv1d_fft","repo":"icantnamemyself/gpht","repo_kind":"official","path":"layers/ETSformer_EncDec.py","file_url":"https://github.com/icantnamemyself/gpht/blob/HEAD/layers/ETSformer_EncDec.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2af0df7394e58e06"}},{"code_sha256_prefix":"592ea8b254b006db","entry":"get_frequency_modes","repo":"icantnamemyself/gpht","repo_kind":"official","path":"layers/FourierCorrelation.py","file_url":"https://github.com/icantnamemyself/gpht/blob/HEAD/layers/FourierCorrelation.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"592ea8b254b006db"}},{"code_sha256_prefix":"f32036738135c8a8","entry":"get_phi_psi","repo":"icantnamemyself/gpht","repo_kind":"official","path":"layers/MultiWaveletCorrelation.py","file_url":"https://github.com/icantnamemyself/gpht/blob/HEAD/layers/MultiWaveletCorrelation.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f32036738135c8a8"}},{"code_sha256_prefix":"4ef26472da51e3aa","entry":"legendreDer","repo":"icantnamemyself/gpht","repo_kind":"official","path":"layers/MultiWaveletCorrelation.py","file_url":"https://github.com/icantnamemyself/gpht/blob/HEAD/layers/MultiWaveletCorrelation.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4ef26472da51e3aa"}},{"code_sha256_prefix":"a54c8c5c47c6a8b8","entry":"phi_","repo":"icantnamemyself/gpht","repo_kind":"official","path":"layers/MultiWaveletCorrelation.py","file_url":"https://github.com/icantnamemyself/gpht/blob/HEAD/layers/MultiWaveletCorrelation.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a54c8c5c47c6a8b8"}},{"code_sha256_prefix":"fd61ceb881a058a2","entry":"get_mask","repo":"icantnamemyself/gpht","repo_kind":"official","path":"layers/Pyraformer_EncDec.py","file_url":"https://github.com/icantnamemyself/gpht/blob/HEAD/layers/Pyraformer_EncDec.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fd61ceb881a058a2"}},{"code_sha256_prefix":"05bdfb126191dbb0","entry":"refer_points","repo":"icantnamemyself/gpht","repo_kind":"official","path":"layers/Pyraformer_EncDec.py","file_url":"https://github.com/icantnamemyself/gpht/blob/HEAD/layers/Pyraformer_EncDec.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"05bdfb126191dbb0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}