{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/timecma-towards-llm-empowered-time-series","title":"TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment","arxiv_id":"2406.01638","date":"2024-06-03","proceeding":null,"authors":["Chenxi Liu","Qianxiong Xu","Hao Miao","Sun Yang","Lingzheng Zhang","Cheng Long","Ziyue Li","Rui Zhao"],"abstract":"Multivariate time series forecasting (MTSF) aims to learn temporal dynamics among variables to forecast future time series. Existing statistical and deep learning-based methods suffer from limited learnable parameters and small-scale training data. Recently, large language models (LLMs) combining time series with textual prompts have achieved promising performance in MTSF. However, we discovered that current LLM-based solutions fall short in learning disentangled embeddings. We introduce TimeCMA, an intuitive yet effective framework for MTSF via cross-modality alignment. Specifically, we present a dual-modality encoding with two branches: the time series encoding branch extracts disentangled yet weak time series embeddings, and the LLM-empowered encoding branch wraps the same time series with text as prompts to obtain entangled yet robust prompt embeddings. As a result, such a cross-modality alignment retrieves both disentangled and robust time series embeddings, \"the best of two worlds\", from the prompt embeddings based on time series and prompt modality similarities. As another key design, to reduce the computational costs from time series with their length textual prompts, we design an effective prompt to encourage the most essential temporal information to be encapsulated in the last token: only the last token is passed to downstream prediction. We further store the last token embeddings to accelerate inference speed. Extensive experiments on eight real datasets demonstrate that TimeCMA outperforms state-of-the-arts.","url_abs":"https://arxiv.org/abs/2406.01638v5","url_pdf":"https://arxiv.org/pdf/2406.01638v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"timecma-towards-llm-empowered-time-series","repo_url":"https://github.com/chenxiliu-hnu/timecma","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"timecma-towards-llm-empowered-time-series","repo_url":"https://github.com/ChenxiLiu-HNU/ST-LLM","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"timecma-towards-llm-empowered-time-series","repo_url":"https://github.com/chenxiliu-hnu/cm2ts","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"multivariate-time-series-forecasting","task_name":"Multivariate Time Series Forecasting"},{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series-forecasting","task_name":"Time Series Forecasting"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/time-series-forecasting-on-etth1-336-1","task":"Time Series Forecasting","dataset":"ETTh1 (336) Multivariate","model":"TimeCMA","rank_in_archive_order":8,"of":72,"metrics":{"MAE":"0.405","MSE":"0.403"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2406.01638","atlas_url":"https://app.syntology.ai/?focus=2406.01638","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.01638"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chenxiliu-hnu/timecma","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chenxiliu-hnu/cm2ts","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ChenxiLiu-HNU/ST-LLM","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_honours":1,"ran":1,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"16b9b78d429636b7","entry":"PositionalEncoding","repo":"chenxiliu-hnu/timecma","repo_kind":"official","path":"layers/TS_Pos_Enc.py","file_url":"https://github.com/chenxiliu-hnu/timecma/blob/HEAD/layers/TS_Pos_Enc.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"16b9b78d429636b7"}},{"code_sha256_prefix":"72b9272542f279c6","entry":"get_activation_fn","repo":"chenxiliu-hnu/timecma","repo_kind":"official","path":"layers/TS_Pos_Enc.py","file_url":"https://github.com/chenxiliu-hnu/timecma/blob/HEAD/layers/TS_Pos_Enc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"72b9272542f279c6"}},{"code_sha256_prefix":"c3fc6992d1e467f3","entry":"Coord2dPosEncoding","repo":"chenxiliu-hnu/timecma","repo_kind":"official","path":"layers/TS_Pos_Enc.py","file_url":"https://github.com/chenxiliu-hnu/timecma/blob/HEAD/layers/TS_Pos_Enc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c3fc6992d1e467f3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}