{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/time-mmd-a-new-multi-domain-multimodal","title":"Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysis","arxiv_id":"2406.08627","date":"2024-06-12","proceeding":null,"authors":["Haoxin Liu","Shangqing Xu","Zhiyuan Zhao","Lingkai Kong","Harshavardhan Kamarthi","Aditya B. Sasanur","Megha Sharma","Jiaming Cui","Qingsong Wen","Chao Zhang","B. Aditya Prakash"],"abstract":"Time series data are ubiquitous across a wide range of real-world domains. While real-world time series analysis (TSA) requires human experts to integrate numerical series data with multimodal domain-specific knowledge, most existing TSA models rely solely on numerical data, overlooking the significance of information beyond numerical series. This oversight is due to the untapped potential of textual series data and the absence of a comprehensive, high-quality multimodal dataset. To overcome this obstacle, we introduce Time-MMD, the first multi-domain, multimodal time series dataset covering 9 primary data domains. Time-MMD ensures fine-grained modality alignment, eliminates data contamination, and provides high usability. Additionally, we develop MM-TSFlib, the first-cut multimodal time-series forecasting (TSF) library, seamlessly pipelining multimodal TSF evaluations based on Time-MMD for in-depth analyses. Extensive experiments conducted on Time-MMD through MM-TSFlib demonstrate significant performance enhancements by extending unimodal TSF to multimodality, evidenced by over 15% mean squared error reduction in general, and up to 40% in domains with rich textual data. More importantly, our datasets and library revolutionize broader applications, impacts, research topics to advance TSA. The dataset is available at https://github.com/AdityaLab/Time-MMD.","url_abs":"https://arxiv.org/abs/2406.08627v4","url_pdf":"https://arxiv.org/pdf/2406.08627v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"time-mmd-a-new-multi-domain-multimodal","repo_url":"https://github.com/adityalab/time-mmd","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"time-mmd-a-new-multi-domain-multimodal","repo_url":"https://github.com/adityalab/mm-tsflib","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series","task_name":"Time Series Analysis"},{"task_slug":"time-series-forecasting","task_name":"Time Series Forecasting"}],"methods":[{"method_slug":null,"method_name":"Library"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2406.08627","atlas_url":"https://app.syntology.ai/?focus=2406.08627","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.08627"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/thuml/Time-Series-Library","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/AdityaLab/Time-MMD","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/adityalab/time-mmd","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/adityalab/mm-tsflib","reach":null}],"summary":{"ran":2,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"173aed9517148a0a","entry":"DFT_series_decomp","repo":"adityalab/mm-tsflib","repo_kind":"official","path":"models/TimeMixer.py","file_url":"https://github.com/adityalab/mm-tsflib/blob/HEAD/models/TimeMixer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"173aed9517148a0a"}},{"code_sha256_prefix":"1f461c12fa9e76b6","entry":"PastDecomposableMixing","repo":"adityalab/mm-tsflib","repo_kind":"official","path":"models/TimeMixer.py","file_url":"https://github.com/adityalab/mm-tsflib/blob/HEAD/models/TimeMixer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1f461c12fa9e76b6"}},{"code_sha256_prefix":"004f481be928afb9","entry":"Model","repo":"adityalab/mm-tsflib","repo_kind":"official","path":"models/TimeMixer.py","file_url":"https://github.com/adityalab/mm-tsflib/blob/HEAD/models/TimeMixer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"004f481be928afb9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}