{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/totem-tokenized-time-series-embeddings-for","title":"TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysis","arxiv_id":"2402.16412","date":"2024-02-26","proceeding":null,"authors":["Sabera Talukder","Yisong Yue","Georgia Gkioxari"],"abstract":"This work studies the problem of time series analysis with generalist (or foundation) models, which are models trained across many data domains. Drawing inspiration from the widespread success of large language models, we consider the simple strategy of discretely tokenizing time series data drawn from a myriad of datasets via self-supervision, then using the fixed tokenization to solve a variety of tasks across many data domains. Canonically, time series models are either trained on a single dataset or built in a task-specific manner (e.g., a forecasting-only model), where many use patches of time as inputs to the model. As such, performant generalist, discrete representation time series models explored across many tasks are of value. Our method, TOkenized Time Series EMbeddings (TOTEM), produces such generalist time series models with minimal or no fine-tuning while exhibiting strong zero-shot performance. We evaluate TOTEM extensively over nearly 500 experiments on three commonly-studied time series tasks with real-world data: imputation (17 baselines, 12 datasets), anomaly detection (19 baselines, 25 datasets), and forecasting (14 baselines, 12 datasets). We conclude that TOTEM matches or outperforms existing state-of-the-art models in both the canonical specialist setting (i.e., training one model on one domain) as well as the generalist setting (i.e., training a single model on many domains), which demonstrates the efficacy of tokenization for general time series analysis. The open-source implementation is available here: https://github.com/SaberaTalukder/TOTEM; a video summary is available here: https://www.youtube.com/watch?v=OqrCpdb6MJk.","url_abs":"https://arxiv.org/abs/2402.16412v2","url_pdf":"https://arxiv.org/pdf/2402.16412v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"totem-tokenized-time-series-embeddings-for","repo_url":"https://github.com/saberatalukder/totem","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"totem-tokenized-time-series-embeddings-for","repo_url":"https://github.com/WenjieDu/PyPOTS","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"anomaly-detection","task_name":"Anomaly Detection"},{"task_slug":"imputation","task_name":"Imputation"},{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series","task_name":"Time Series Analysis"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.16412","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.16412"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/WenjieDu/PyPOTS","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/saberatalukder/totem","reach":{"status":"ok"}}],"summary":{"ran":11,"unverified":2},"by_repo_kind":{"official":{"samples":13,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":13,"samples":[{"code_sha256_prefix":"55dfde0d12235653","entry":"codes2time","repo":"saberatalukder/totem","repo_kind":"official","path":"forecasting/extract_forecasting_data.py","file_url":"https://github.com/saberatalukder/totem/blob/HEAD/forecasting/extract_forecasting_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"55dfde0d12235653"}},{"code_sha256_prefix":"cb77fdb4a1e324c7","entry":"codes2timerevin","repo":"saberatalukder/totem","repo_kind":"official","path":"anomaly_detection/detect_anomaly.py","file_url":"https://github.com/saberatalukder/totem/blob/HEAD/anomaly_detection/detect_anomaly.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cb77fdb4a1e324c7"}},{"code_sha256_prefix":"762b2be39fdadf5a","entry":"create_datloaders","repo":"saberatalukder/totem","repo_kind":"official","path":"anomaly_detection/detect_anomaly.py","file_url":"https://github.com/saberatalukder/totem/blob/HEAD/anomaly_detection/detect_anomaly.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"762b2be39fdadf5a"}},{"code_sha256_prefix":"b5c7d28a17f662a4","entry":"create_datloaders","repo":"saberatalukder/totem","repo_kind":"official","path":"anomaly_detection/train_vqvae.py","file_url":"https://github.com/saberatalukder/totem/blob/HEAD/anomaly_detection/train_vqvae.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b5c7d28a17f662a4"}},{"code_sha256_prefix":"1a119a859ec7ea04","entry":"create_time_series_dataloader","repo":"saberatalukder/totem","repo_kind":"official","path":"forecasting/generalist_eval.py","file_url":"https://github.com/saberatalukder/totem/blob/HEAD/forecasting/generalist_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1a119a859ec7ea04"}},{"code_sha256_prefix":"96092138a8882ebc","entry":"get_params","repo":"saberatalukder/totem","repo_kind":"official","path":"forecasting/generalist_eval.py","file_url":"https://github.com/saberatalukder/totem/blob/HEAD/forecasting/generalist_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"96092138a8882ebc"}},{"code_sha256_prefix":"af1552f9da41a7cc","entry":"load_and_flatten","repo":"saberatalukder/totem","repo_kind":"official","path":"anomaly_detection/combine_datasets.py","file_url":"https://github.com/saberatalukder/totem/blob/HEAD/anomaly_detection/combine_datasets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"af1552f9da41a7cc"}},{"code_sha256_prefix":"7302b9f7263f76d0","entry":"load_and_flatten","repo":"saberatalukder/totem","repo_kind":"official","path":"anomaly_detection/combine_labels.py","file_url":"https://github.com/saberatalukder/totem/blob/HEAD/anomaly_detection/combine_labels.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7302b9f7263f76d0"}},{"code_sha256_prefix":"67b566e76ba373ba","entry":"revintime2codes","repo":"saberatalukder/totem","repo_kind":"official","path":"anomaly_detection/detect_anomaly.py","file_url":"https://github.com/saberatalukder/totem/blob/HEAD/anomaly_detection/detect_anomaly.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"67b566e76ba373ba"}},{"code_sha256_prefix":"92447a783743bb61","entry":"save_data","repo":"saberatalukder/totem","repo_kind":"official","path":"anomaly_detection/save_chunked_data.py","file_url":"https://github.com/saberatalukder/totem/blob/HEAD/anomaly_detection/save_chunked_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"92447a783743bb61"}},{"code_sha256_prefix":"99487b1b03cba02c","entry":"time2codes","repo":"saberatalukder/totem","repo_kind":"official","path":"forecasting/extract_forecasting_data.py","file_url":"https://github.com/saberatalukder/totem/blob/HEAD/forecasting/extract_forecasting_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"99487b1b03cba02c"}},{"code_sha256_prefix":"7b15d87455a71d97","entry":"inference","repo":"saberatalukder/totem","repo_kind":"official","path":"forecasting/generalist_eval.py","file_url":"https://github.com/saberatalukder/totem/blob/HEAD/forecasting/generalist_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7b15d87455a71d97"}},{"code_sha256_prefix":"b299340c6f48dc63","entry":"train_model","repo":"saberatalukder/totem","repo_kind":"official","path":"anomaly_detection/train_vqvae.py","file_url":"https://github.com/saberatalukder/totem/blob/HEAD/anomaly_detection/train_vqvae.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b299340c6f48dc63"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}