{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/federated-foundation-models-on-heterogeneous","title":"Federated Foundation Models on Heterogeneous Time Series","arxiv_id":"2412.08906","date":"2024-12-12","proceeding":null,"authors":["Shengchao Chen","Guodong Long","Jing Jiang","Chengqi Zhang"],"abstract":"Training a general-purpose time series foundation models with robust generalization capabilities across diverse applications from scratch is still an open challenge. Efforts are primarily focused on fusing cross-domain time series datasets to extract shared subsequences as tokens for training models on Transformer architecture. However, due to significant statistical heterogeneity across domains, this cross-domain fusing approach doesn't work effectively as the same as fusing texts and images. To tackle this challenge, this paper proposes a novel federated learning approach to address the heterogeneity in time series foundation models training, namely FFTS. Specifically, each data-holding organization is treated as an independent client in a collaborative learning framework with federated settings, and then many client-specific local models will be trained to preserve the unique characteristics per dataset. Moreover, a new regularization mechanism will be applied to both client-side and server-side, thus to align the shared knowledge across heterogeneous datasets from different domains. Extensive experiments on benchmark datasets demonstrate the effectiveness of the proposed federated learning approach. The newly learned time series foundation models achieve superior generalization capabilities on cross-domain time series analysis tasks, including forecasting, imputation, and anomaly detection.","url_abs":"https://arxiv.org/abs/2412.08906v1","url_pdf":"https://arxiv.org/pdf/2412.08906v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"federated-foundation-models-on-heterogeneous","repo_url":"https://github.com/shengchaochen82/FFTS","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"anomaly-detection","task_name":"Anomaly Detection"},{"task_slug":"federated-learning","task_name":"Federated Learning"},{"task_slug":"imputation","task_name":"Imputation"},{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series","task_name":"Time Series Analysis"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2412.08906","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.08906"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shengchaochen82/FFTS","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":2,"unverified":5},"by_repo_kind":{"official":{"samples":7,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2dcb277b78902447","entry":"nanstd","repo":"shengchaochen82/FFTS","repo_kind":"official","path":"data_provider/utils.py","file_url":"https://github.com/shengchaochen82/FFTS/blob/HEAD/data_provider/utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2dcb277b78902447"}},{"code_sha256_prefix":"3dc94406bdfb6cc2","entry":"nanvar","repo":"shengchaochen82/FFTS","repo_kind":"official","path":"data_provider/utils.py","file_url":"https://github.com/shengchaochen82/FFTS/blob/HEAD/data_provider/utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3dc94406bdfb6cc2"}},{"code_sha256_prefix":"3ebe34eeba1647ec","entry":"convert_tsf_to_dataframe","repo":"shengchaochen82/FFTS","repo_kind":"official","path":"data_provider/monash_data.py","file_url":"https://github.com/shengchaochen82/FFTS/blob/HEAD/data_provider/monash_data.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3ebe34eeba1647ec"}},{"code_sha256_prefix":"bdc787c888ab83a2","entry":"divide_no_nan","repo":"shengchaochen82/FFTS","repo_kind":"official","path":"utils/losses.py","file_url":"https://github.com/shengchaochen82/FFTS/blob/HEAD/utils/losses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bdc787c888ab83a2"}},{"code_sha256_prefix":"01ee4d87363ba62e","entry":"interpolate_timeseries","repo":"shengchaochen82/FFTS","repo_kind":"official","path":"data_provider/utils.py","file_url":"https://github.com/shengchaochen82/FFTS/blob/HEAD/data_provider/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"01ee4d87363ba62e"}},{"code_sha256_prefix":"99e2999523c19c2d","entry":"load_filenames_from_json","repo":"shengchaochen82/FFTS","repo_kind":"official","path":"data_provider/load_utils.py","file_url":"https://github.com/shengchaochen82/FFTS/blob/HEAD/data_provider/load_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"99e2999523c19c2d"}},{"code_sha256_prefix":"4ff98237d52255b3","entry":"load_from_tsfile","repo":"shengchaochen82/FFTS","repo_kind":"official","path":"data_provider/monash_data.py","file_url":"https://github.com/shengchaochen82/FFTS/blob/HEAD/data_provider/monash_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4ff98237d52255b3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}