{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/himtm-hierarchical-multi-scale-masked-time","title":"HiMTM: Hierarchical Multi-Scale Masked Time Series Modeling for Long-Term Forecasting","arxiv_id":"2401.05012","date":"2024-01-10","proceeding":null,"authors":["Shubao Zhao","Ming Jin","Zhaoxiang Hou","Chengyi Yang","Zengxiang Li","Qingsong Wen","Yi Wang"],"abstract":"Time series forecasting is crucial and challenging in the real world. The recent surge in interest regarding time series foundation models, which cater to a diverse array of downstream tasks, is noteworthy. However, existing methods often overlook the multi-scale nature of time series, an aspect crucial for precise forecasting. To bridge this gap, we propose HiMTM, a hierarchical multi-scale masked time series modeling method designed for long-term forecasting. Specifically, it comprises four integral components: (1) hierarchical multi-scale transformer (HMT) to capture temporal information at different scales; (2) decoupled encoder-decoder (DED) forces the encoder to focus on feature extraction, while the decoder to focus on pretext tasks; (3) multi-scale masked reconstruction (MMR) provides multi-stage supervision signals for pre-training; (4) cross-scale attention fine-tuning (CSA-FT) to capture dependencies between different scales for forecasting. Collectively, these components enhance multi-scale feature extraction capabilities in masked time series modeling and contribute to improved prediction accuracy. We conduct extensive experiments on 7 mainstream datasets to prove that HiMTM has obvious advantages over contemporary self-supervised and end-to-end learning methods. The effectiveness of HiMTM is further showcased by its application in the industry of natural gas demand forecasting.","url_abs":"https://arxiv.org/abs/2401.05012v1","url_pdf":"https://arxiv.org/pdf/2401.05012v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"demand-forecasting","task_name":"Demand Forecasting"},{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series-forecasting","task_name":"Time Series Forecasting"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/time-series-forecasting-on-etth1-336-1","task":"Time Series Forecasting","dataset":"ETTh1 (336) Multivariate","model":"HiMTM","rank_in_archive_order":19,"of":72,"metrics":{"MAE":"0.430","MSE":"0.422"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2401.05012","atlas_url":"https://app.syntology.ai/?focus=2401.05012","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}