{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multi-modality-spatio-temporal-forecasting","title":"Multi-Modality Spatio-Temporal Forecasting via Self-Supervised Learning","arxiv_id":"2405.03255","date":"2024-05-06","proceeding":null,"authors":["Jiewen Deng","Renhe Jiang","JiaQi Zhang","Xuan Song"],"abstract":"Multi-modality spatio-temporal (MoST) data extends spatio-temporal (ST) data by incorporating multiple modalities, which is prevalent in monitoring systems, encompassing diverse traffic demands and air quality assessments. Despite significant strides in ST modeling in recent years, there remains a need to emphasize harnessing the potential of information from different modalities. Robust MoST forecasting is more challenging because it possesses (i) high-dimensional and complex internal structures and (ii) dynamic heterogeneity caused by temporal, spatial, and modality variations. In this study, we propose a novel MoST learning framework via Self-Supervised Learning, namely MoSSL, which aims to uncover latent patterns from temporal, spatial, and modality perspectives while quantifying dynamic heterogeneity. Experiment results on two real-world MoST datasets verify the superiority of our approach compared with the state-of-the-art baselines. Model implementation is available at https://github.com/beginner-sketch/MoSSL.","url_abs":"https://arxiv.org/abs/2405.03255v1","url_pdf":"https://arxiv.org/pdf/2405.03255v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multi-modality-spatio-temporal-forecasting","repo_url":"https://github.com/beginner-sketch/mossl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"spatio-temporal-forecasting","task_name":"Spatio-Temporal Forecasting"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.03255","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.03255"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/beginner-sketch/mossl","reach":null}],"summary":{"ran":7,"unverified":1},"by_repo_kind":{"official":{"samples":8,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"294007eb0a8c497f","entry":"GSSL","repo":"beginner-sketch/mossl","repo_kind":"official","path":"models/MoSSL.py","file_url":"https://github.com/beginner-sketch/mossl/blob/HEAD/models/MoSSL.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"294007eb0a8c497f"}},{"code_sha256_prefix":"ab6954e04f03bd6f","entry":"MA","repo":"beginner-sketch/mossl","repo_kind":"official","path":"models/MoSSL.py","file_url":"https://github.com/beginner-sketch/mossl/blob/HEAD/models/MoSSL.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ab6954e04f03bd6f"}},{"code_sha256_prefix":"8a1e48f45ee33c62","entry":"MDA","repo":"beginner-sketch/mossl","repo_kind":"official","path":"models/MoSSL.py","file_url":"https://github.com/beginner-sketch/mossl/blob/HEAD/models/MoSSL.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8a1e48f45ee33c62"}},{"code_sha256_prefix":"47319c2abca45f52","entry":"MSSL","repo":"beginner-sketch/mossl","repo_kind":"official","path":"models/MoSSL.py","file_url":"https://github.com/beginner-sketch/mossl/blob/HEAD/models/MoSSL.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"47319c2abca45f52"}},{"code_sha256_prefix":"0d5f0e1f3698f28b","entry":"MoST_Encoder","repo":"beginner-sketch/mossl","repo_kind":"official","path":"models/MoSSL.py","file_url":"https://github.com/beginner-sketch/mossl/blob/HEAD/models/MoSSL.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0d5f0e1f3698f28b"}},{"code_sha256_prefix":"866dff57161742cb","entry":"ResidualBlock","repo":"beginner-sketch/mossl","repo_kind":"official","path":"models/MoSSL.py","file_url":"https://github.com/beginner-sketch/mossl/blob/HEAD/models/MoSSL.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"866dff57161742cb"}},{"code_sha256_prefix":"460c42976a5acad3","entry":"SA","repo":"beginner-sketch/mossl","repo_kind":"official","path":"models/MoSSL.py","file_url":"https://github.com/beginner-sketch/mossl/blob/HEAD/models/MoSSL.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"460c42976a5acad3"}},{"code_sha256_prefix":"27bd9ae0f1a5e02d","entry":"MoSSL","repo":"beginner-sketch/mossl","repo_kind":"official","path":"models/MoSSL.py","file_url":"https://github.com/beginner-sketch/mossl/blob/HEAD/models/MoSSL.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"27bd9ae0f1a5e02d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}