{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/imts-is-worth-time-times-channel-patches","title":"IMTS is Worth Time $\\times$ Channel Patches: Visual Masked Autoencoders for Irregular Multivariate Time Series Prediction","arxiv_id":"2505.22815","date":"2025-05-28","proceeding":null,"authors":["Zhangyi Hu","Jiemin Wu","Hua Xu","Mingqian Liao","Ninghui Feng","Bo Gao","Songning Lai","Yutao Yue"],"abstract":"Irregular Multivariate Time Series (IMTS) forecasting is challenging due to the unaligned nature of multi-channel signals and the prevalence of extensive missing data. Existing methods struggle to capture reliable temporal patterns from such data due to significant missing values. While pre-trained foundation models show potential for addressing these challenges, they are typically designed for Regularly Sampled Time Series (RTS). Motivated by the visual Mask AutoEncoder's (MAE) powerful capability for modeling sparse multi-channel information and its success in RTS forecasting, we propose VIMTS, a framework adapting Visual MAE for IMTS forecasting. To mitigate the effect of missing values, VIMTS first processes IMTS along the timeline into feature patches at equal intervals. These patches are then complemented using learned cross-channel dependencies. Then it leverages visual MAE's capability in handling sparse multichannel data for patch reconstruction, followed by a coarse-to-fine technique to generate precise predictions from focused contexts. In addition, we integrate self-supervised learning for improved IMTS modeling by adapting the visual MAE to IMTS data. Extensive experiments demonstrate VIMTS's superior performance and few-shot capability, advancing the application of visual foundation models in more general time series tasks. Our code is available at https://github.com/WHU-HZY/VIMTS.","url_abs":"https://arxiv.org/abs/2505.22815v1","url_pdf":"https://arxiv.org/pdf/2505.22815v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"imts-is-worth-time-times-channel-patches","repo_url":"https://github.com/whu-hzy/vimts","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"missing-values","task_name":"Missing Values"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series-prediction","task_name":"Time Series Prediction"}],"methods":[{"method_slug":"mae","method_name":"MAE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2505.22815","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.22815"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/WHU-HZY/VIMTS","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/whu-hzy/vimts","reach":{"status":"ok"}}],"summary":{"ran":4,"ran_fixture":1,"unverified":4},"by_repo_kind":{"official":{"samples":9,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"4943729c88110c21","entry":"IMTS_PatchEmbed","repo":"whu-hzy/vimts","repo_kind":"official","path":"IMTS/lib/models/visionts/models_mae.py","file_url":"https://github.com/whu-hzy/vimts/blob/HEAD/IMTS/lib/models/visionts/models_mae.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4943729c88110c21"}},{"code_sha256_prefix":"134faf93427daea3","entry":"Linear","repo":"whu-hzy/vimts","repo_kind":"official","path":"IMTS/lib/models/visionts/models_mae.py","file_url":"https://github.com/whu-hzy/vimts/blob/HEAD/IMTS/lib/models/visionts/models_mae.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"134faf93427daea3"}},{"code_sha256_prefix":"7e045c902914c785","entry":"Mlp","repo":"whu-hzy/vimts","repo_kind":"official","path":"IMTS/lib/models/visionts/models_mae.py","file_url":"https://github.com/whu-hzy/vimts/blob/HEAD/IMTS/lib/models/visionts/models_mae.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7e045c902914c785"}},{"code_sha256_prefix":"be2cd94cfe0ee2f0","entry":"get_2d_sincos_pos_embed_for_seq","repo":"whu-hzy/vimts","repo_kind":"official","path":"IMTS/lib/models/visionts/models_mae.py","file_url":"https://github.com/whu-hzy/vimts/blob/HEAD/IMTS/lib/models/visionts/models_mae.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"be2cd94cfe0ee2f0"}},{"code_sha256_prefix":"d47294d8704c58f9","entry":"nconv","repo":"whu-hzy/vimts","repo_kind":"official","path":"IMTS/lib/models/visionts/models_mae.py","file_url":"https://github.com/whu-hzy/vimts/blob/HEAD/IMTS/lib/models/visionts/models_mae.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d47294d8704c58f9"}},{"code_sha256_prefix":"2698a4c6a1a414f9","entry":"Attention","repo":"whu-hzy/vimts","repo_kind":"official","path":"IMTS/lib/models/visionts/models_mae.py","file_url":"https://github.com/whu-hzy/vimts/blob/HEAD/IMTS/lib/models/visionts/models_mae.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2698a4c6a1a414f9"}},{"code_sha256_prefix":"f4ee7167fad69080","entry":"Block","repo":"whu-hzy/vimts","repo_kind":"official","path":"IMTS/lib/models/visionts/models_mae.py","file_url":"https://github.com/whu-hzy/vimts/blob/HEAD/IMTS/lib/models/visionts/models_mae.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f4ee7167fad69080"}},{"code_sha256_prefix":"0c4169c946e2736d","entry":"MaskedAutoencoderViT","repo":"whu-hzy/vimts","repo_kind":"official","path":"IMTS/lib/models/visionts/models_mae.py","file_url":"https://github.com/whu-hzy/vimts/blob/HEAD/IMTS/lib/models/visionts/models_mae.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0c4169c946e2736d"}},{"code_sha256_prefix":"319f1727947e81b0","entry":"gcn","repo":"whu-hzy/vimts","repo_kind":"official","path":"IMTS/lib/models/visionts/models_mae.py","file_url":"https://github.com/whu-hzy/vimts/blob/HEAD/IMTS/lib/models/visionts/models_mae.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"319f1727947e81b0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}