{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mixture-of-linear-experts-for-long-term-time","title":"Mixture-of-Linear-Experts for Long-term Time Series Forecasting","arxiv_id":"2312.06786","date":"2023-12-11","proceeding":null,"authors":["Ronghao Ni","Zinan Lin","Shuaiqi Wang","Giulia Fanti"],"abstract":"Long-term time series forecasting (LTSF) aims to predict future values of a time series given the past values. The current state-of-the-art (SOTA) on this problem is attained in some cases by linear-centric models, which primarily feature a linear mapping layer. However, due to their inherent simplicity, they are not able to adapt their prediction rules to periodic changes in time series patterns. To address this challenge, we propose a Mixture-of-Experts-style augmentation for linear-centric models and propose Mixture-of-Linear-Experts (MoLE). Instead of training a single model, MoLE trains multiple linear-centric models (i.e., experts) and a router model that weighs and mixes their outputs. While the entire framework is trained end-to-end, each expert learns to specialize in a specific temporal pattern, and the router model learns to compose the experts adaptively. Experiments show that MoLE reduces forecasting error of linear-centric models, including DLinear, RLinear, and RMLP, in over 78% of the datasets and settings we evaluated. By using MoLE existing linear-centric models can achieve SOTA LTSF results in 68% of the experiments that PatchTST reports and we compare to, whereas existing single-head linear-centric models achieve SOTA results in only 25% of cases.","url_abs":"https://arxiv.org/abs/2312.06786v3","url_pdf":"https://arxiv.org/pdf/2312.06786v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mixture-of-linear-experts-for-long-term-time","repo_url":"https://github.com/rogerni/mole","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"mixture-of-experts","task_name":"Mixture-of-Experts"},{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series-forecasting","task_name":"Time Series Forecasting"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/time-series-forecasting-on-etth1-192-1","task":"Time Series Forecasting","dataset":"ETTh1 (192) Multivariate","model":"MoLE-RLinear","rank_in_archive_order":8,"of":17,"metrics":{"MSE":"0.403"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth1-192-1","task":"Time Series Forecasting","dataset":"ETTh1 (192) Multivariate","model":"MoLE-DLinear","rank_in_archive_order":17,"of":17,"metrics":{"MSE":"0.453"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth1-336-1","task":"Time Series Forecasting","dataset":"ETTh1 (336) Multivariate","model":"MoLE-RLinear","rank_in_archive_order":30,"of":72,"metrics":{"MSE":"0.43"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth1-336-1","task":"Time Series Forecasting","dataset":"ETTh1 (336) Multivariate","model":"MoLE-DLinear","rank_in_archive_order":50,"of":72,"metrics":{"MSE":"0.469"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth1-720-1","task":"Time Series Forecasting","dataset":"ETTh1 (720) Multivariate","model":"MoLE-RLinear","rank_in_archive_order":11,"of":22,"metrics":{"MSE":"0.449"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth1-720-1","task":"Time Series Forecasting","dataset":"ETTh1 (720) Multivariate","model":"MoLE-DLinear","rank_in_archive_order":17,"of":22,"metrics":{"MSE":"0.505"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth1-96-1","task":"Time Series Forecasting","dataset":"ETTh1 (96) Multivariate","model":"MoLE-RLinear","rank_in_archive_order":13,"of":15,"metrics":{"MSE":"0.375"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth1-96-1","task":"Time Series Forecasting","dataset":"ETTh1 (96) Multivariate","model":"MoLE-DLinear","rank_in_archive_order":14,"of":15,"metrics":{"MSE":"0.377"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth2-192-1","task":"Time Series Forecasting","dataset":"ETTh2 (192) Multivariate","model":"MoLE-RLinear","rank_in_archive_order":10,"of":16,"metrics":{"MSE":"0.336"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth2-192-1","task":"Time Series Forecasting","dataset":"ETTh2 (192) Multivariate","model":"MoLE-DLinear","rank_in_archive_order":14,"of":16,"metrics":{"MSE":"0.362"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth2-336-1","task":"Time Series Forecasting","dataset":"ETTh2 (336) Multivariate","model":"MoLE-RLinear","rank_in_archive_order":13,"of":20,"metrics":{"MSE":"0.371"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth2-336-1","task":"Time Series Forecasting","dataset":"ETTh2 (336) Multivariate","model":"MoLE-DLinear","rank_in_archive_order":15,"of":20,"metrics":{"MSE":"0.419"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth2-720-1","task":"Time Series Forecasting","dataset":"ETTh2 (720) Multivariate","model":"MoLE-RLinear","rank_in_archive_order":11,"of":20,"metrics":{"MSE":"0.409"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth2-720-1","task":"Time Series Forecasting","dataset":"ETTh2 (720) Multivariate","model":"MoLE-DLinear","rank_in_archive_order":17,"of":20,"metrics":{"MSE":"0.605"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth2-96-1","task":"Time Series Forecasting","dataset":"ETTh2 (96) Multivariate","model":"MoLE-RLinear","rank_in_archive_order":8,"of":16,"metrics":{"MSE":"0.273"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth2-96-1","task":"Time Series Forecasting","dataset":"ETTh2 (96) Multivariate","model":"MoLE-DLinear","rank_in_archive_order":14,"of":16,"metrics":{"MSE":"0.287"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-ettm1-192-1","task":"Time Series Forecasting","dataset":"ETTm1 (192) Multivariate","model":"MoLE-DLinear","rank_in_archive_order":5,"of":9,"metrics":{"MSE":"0.328"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-ettm1-336-1","task":"Time Series Forecasting","dataset":"ETTm1 (336) Multivariate","model":"MoLE-DLinear","rank_in_archive_order":7,"of":8,"metrics":{"MSE":"0.38"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-ettm1-720-1","task":"Time Series Forecasting","dataset":"ETTm1 (720) Multivariate","model":"MoLE-DLinear","rank_in_archive_order":7,"of":8,"metrics":{"MSE":"0.447"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-ettm1-96-1","task":"Time Series Forecasting","dataset":"ETTm1 (96) Multivariate","model":"MoLE-DLinear","rank_in_archive_order":4,"of":9,"metrics":{"MSE":"0.286"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-ettm2-192-1","task":"Time Series Forecasting","dataset":"ETTm2 (192) Multivariate","model":"MoLE-DLinear","rank_in_archive_order":7,"of":9,"metrics":{"MSE":"0.233"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-ettm2-336-1","task":"Time Series Forecasting","dataset":"ETTm2 (336) Multivariate","model":"MoLE-DLinear","rank_in_archive_order":7,"of":8,"metrics":{"MSE":"0.289"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-ettm2-720-1","task":"Time Series Forecasting","dataset":"ETTm2 (720) Multivariate","model":"MoLE-DLinear","rank_in_archive_order":7,"of":8,"metrics":{"MSE":"0.399"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-ettm2-96-1","task":"Time Series Forecasting","dataset":"ETTm2 (96) Multivariate","model":"MoLE-DLinear","rank_in_archive_order":8,"of":9,"metrics":{"MSE":"0.168"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-electricity-192","task":"Time Series Forecasting","dataset":"Electricity (192)","model":"MoLE-DLinear","rank_in_archive_order":3,"of":8,"metrics":{"MSE":"0.147"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-electricity-336","task":"Time Series Forecasting","dataset":"Electricity (336)","model":"MoLE-DLinear","rank_in_archive_order":4,"of":9,"metrics":{"MSE":"0.162"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-electricity-720","task":"Time Series Forecasting","dataset":"Electricity (720)","model":"MoLE-RMLP","rank_in_archive_order":1,"of":8,"metrics":{"MSE":"0.178"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-electricity-720","task":"Time Series Forecasting","dataset":"Electricity (720)","model":"MoLE-DLinear","rank_in_archive_order":2,"of":8,"metrics":{"MSE":"0.18"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-electricity-96","task":"Time Series Forecasting","dataset":"Electricity (96)","model":"MoLE-RMLP","rank_in_archive_order":4,"of":11,"metrics":{"MSE":"0.129"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-electricity-96","task":"Time Series Forecasting","dataset":"Electricity (96)","model":"MoLE-DLinear","rank_in_archive_order":7,"of":11,"metrics":{"MSE":"0.131"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather-192","task":"Time Series Forecasting","dataset":"Weather (192)","model":"MoLE-RMLP","rank_in_archive_order":7,"of":13,"metrics":{"MSE":"0.19"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather-192","task":"Time Series Forecasting","dataset":"Weather (192)","model":"MoLE-DLinear","rank_in_archive_order":10,"of":13,"metrics":{"MSE":"0.203"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather-336","task":"Time Series Forecasting","dataset":"Weather (336)","model":"MoLE-DLinear","rank_in_archive_order":4,"of":11,"metrics":{"MSE":"0.238"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather-720","task":"Time Series Forecasting","dataset":"Weather (720)","model":"MoLE-DLinear","rank_in_archive_order":5,"of":11,"metrics":{"MSE":"0.314"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather-96","task":"Time Series Forecasting","dataset":"Weather (96)","model":"MoLE-DLinear","rank_in_archive_order":8,"of":12,"metrics":{"MSE":"0.147"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k114-192","task":"Time Series Forecasting","dataset":"Weather2K114 (192)","model":"MoLE-DLinear","rank_in_archive_order":1,"of":1,"metrics":{"MSE":"0.405"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k114-336","task":"Time Series Forecasting","dataset":"Weather2K114 (336)","model":"MoLE-DLinear","rank_in_archive_order":1,"of":1,"metrics":{"MSE":"0.415"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k114-720","task":"Time Series Forecasting","dataset":"Weather2K114 (720)","model":"MoLE-DLinear","rank_in_archive_order":1,"of":1,"metrics":{"MSE":"0.425"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k114-96","task":"Time Series Forecasting","dataset":"Weather2K114 (96)","model":"MoLE-DLinear","rank_in_archive_order":1,"of":1,"metrics":{"MSE":"0.391"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k1786-192","task":"Time Series Forecasting","dataset":"Weather2K1786 (192)","model":"MoLE-RLinear","rank_in_archive_order":1,"of":2,"metrics":{"MSE":"0.581"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k1786-192","task":"Time Series Forecasting","dataset":"Weather2K1786 (192)","model":"MoLE-DLinear","rank_in_archive_order":2,"of":2,"metrics":{"MSE":"0.601"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k1786-336","task":"Time Series Forecasting","dataset":"Weather2K1786 (336)","model":"MoLE-DLinear","rank_in_archive_order":1,"of":1,"metrics":{"MSE":"0.603"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k1786-720","task":"Time Series Forecasting","dataset":"Weather2K1786 (720)","model":"MoLE-RLinear","rank_in_archive_order":1,"of":2,"metrics":{"MSE":"0.628"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k1786-720","task":"Time Series Forecasting","dataset":"Weather2K1786 (720)","model":"MoLE-DLinear","rank_in_archive_order":2,"of":2,"metrics":{"MSE":"0.66"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k1786-96","task":"Time Series Forecasting","dataset":"Weather2K1786 (96)","model":"MoLE-DLinear","rank_in_archive_order":1,"of":2,"metrics":{"MSE":"0.535"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k1786-96","task":"Time Series Forecasting","dataset":"Weather2K1786 (96)","model":"MoLE-RLinear","rank_in_archive_order":2,"of":2,"metrics":{"MSE":"0.535"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k79-192","task":"Time Series Forecasting","dataset":"Weather2K79 (192)","model":"MoLE-DLinear","rank_in_archive_order":1,"of":1,"metrics":{"MSE":"0.566"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k79-336","task":"Time Series Forecasting","dataset":"Weather2K79 (336)","model":"MoLE-DLinear","rank_in_archive_order":1,"of":1,"metrics":{"MSE":"0.546"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k79-720","task":"Time Series Forecasting","dataset":"Weather2K79 (720)","model":"MoLE-DLinear","rank_in_archive_order":1,"of":1,"metrics":{"MSE":"0.535"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k79-96","task":"Time Series Forecasting","dataset":"Weather2K79 (96)","model":"MoLE-DLinear","rank_in_archive_order":1,"of":1,"metrics":{"MSE":"0.555"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k850-192","task":"Time Series Forecasting","dataset":"Weather2K850 (192)","model":"MoLE-DLinear","rank_in_archive_order":1,"of":1,"metrics":{"MSE":"0.484"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k850-336","task":"Time Series Forecasting","dataset":"Weather2K850 (336)","model":"MoLE-DLinear","rank_in_archive_order":1,"of":1,"metrics":{"MSE":"0.474"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k850-720","task":"Time Series Forecasting","dataset":"Weather2K850 (720)","model":"MoLE-DLinear","rank_in_archive_order":1,"of":1,"metrics":{"MSE":"0.461"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k850-96","task":"Time Series Forecasting","dataset":"Weather2K850 (96)","model":"MoLE-RLinear","rank_in_archive_order":1,"of":2,"metrics":{"MSE":"0.471"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-weather2k850-96","task":"Time Series Forecasting","dataset":"Weather2K850 (96)","model":"MoLE-DLinear","rank_in_archive_order":2,"of":2,"metrics":{"MSE":"0.474"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2312.06786","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.06786"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rogerni/mole","reach":{"status":"ok"}}],"summary":{"unverified":1},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"5c037d281a902e51","entry":"augmentation","repo":"rogerni/mole","repo_kind":"official","path":"utils/augmentations.py","file_url":"https://github.com/rogerni/mole/blob/HEAD/utils/augmentations.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5c037d281a902e51"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}