{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tsgbench-time-series-generation-benchmark","title":"TSGBench: Time Series Generation Benchmark","arxiv_id":"2309.03755","date":"2023-09-07","proceeding":null,"authors":["Yihao Ang","Qiang Huang","Yifan Bao","Anthony K. H. Tung","Zhiyong Huang"],"abstract":"Synthetic Time Series Generation (TSG) is crucial in a range of applications, including data augmentation, anomaly detection, and privacy preservation. Although significant strides have been made in this field, existing methods exhibit three key limitations: (1) They often benchmark against similar model types, constraining a holistic view of performance capabilities. (2) The use of specialized synthetic and private datasets introduces biases and hampers generalizability. (3) Ambiguous evaluation measures, often tied to custom networks or downstream tasks, hinder consistent and fair comparison. To overcome these limitations, we introduce \\textsf{TSGBench}, the inaugural Time Series Generation Benchmark, designed for a unified and comprehensive assessment of TSG methods. It comprises three modules: (1) a curated collection of publicly available, real-world datasets tailored for TSG, together with a standardized preprocessing pipeline; (2) a comprehensive evaluation measures suite including vanilla measures, new distance-based assessments, and visualization tools; (3) a pioneering generalization test rooted in Domain Adaptation (DA), compatible with all methods. We have conducted comprehensive experiments using \\textsf{TSGBench} across a spectrum of ten real-world datasets from diverse domains, utilizing ten advanced TSG methods and twelve evaluation measures. The results highlight the reliability and efficacy of \\textsf{TSGBench} in evaluating TSG methods. Crucially, \\textsf{TSGBench} delivers a statistical analysis of the performance rankings of these methods, illuminating their varying performance across different datasets and measures and offering nuanced insights into the effectiveness of each method.","url_abs":"https://arxiv.org/abs/2309.03755v2","url_pdf":"https://arxiv.org/pdf/2309.03755v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tsgbench-time-series-generation-benchmark","repo_url":"https://github.com/yihaoang/tsgbench","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"anomaly-detection","task_name":"Anomaly Detection"},{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"domain-adaptation","task_name":"Domain Adaptation"},{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series-generation","task_name":"Time Series Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2309.03755","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.03755"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yihaoang/tsgbench","reach":{"status":"ok"}}],"summary":{"ran":9,"ran_violates":2,"unverified":3},"by_repo_kind":{"official":{"samples":14,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"12d73e40361cea46","entry":"acf_torch","repo":"yihaoang/tsgbench","repo_kind":"official","path":"src/feature_based_measures.py","file_url":"https://github.com/yihaoang/tsgbench/blob/HEAD/src/feature_based_measures.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"12d73e40361cea46"}},{"code_sha256_prefix":"c682e0fb99675e31","entry":"calculate_ed","repo":"yihaoang/tsgbench","repo_kind":"official","path":"src/evaluation.py","file_url":"https://github.com/yihaoang/tsgbench/blob/HEAD/src/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c682e0fb99675e31"}},{"code_sha256_prefix":"7fedf1b6457b251e","entry":"calculate_fid","repo":"yihaoang/tsgbench","repo_kind":"official","path":"src/evaluation.py","file_url":"https://github.com/yihaoang/tsgbench/blob/HEAD/src/evaluation.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7fedf1b6457b251e"}},{"code_sha256_prefix":"5d976181dd5ef419","entry":"calculate_mdd","repo":"yihaoang/tsgbench","repo_kind":"official","path":"src/feature_based_measures.py","file_url":"https://github.com/yihaoang/tsgbench/blob/HEAD/src/feature_based_measures.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5d976181dd5ef419"}},{"code_sha256_prefix":"b11fd8459d71a086","entry":"determine_device","repo":"yihaoang/tsgbench","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/yihaoang/tsgbench/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b11fd8459d71a086"}},{"code_sha256_prefix":"4b3751ed50cd6049","entry":"find_length","repo":"yihaoang/tsgbench","repo_kind":"official","path":"src/preprocess.py","file_url":"https://github.com/yihaoang/tsgbench/blob/HEAD/src/preprocess.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4b3751ed50cd6049"}},{"code_sha256_prefix":"fd10054ec4772b80","entry":"generate_binomial_mask","repo":"yihaoang/tsgbench","repo_kind":"official","path":"src/ts2vec.py","file_url":"https://github.com/yihaoang/tsgbench/blob/HEAD/src/ts2vec.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fd10054ec4772b80"}},{"code_sha256_prefix":"acda587a1d1be3bf","entry":"generate_continuous_mask","repo":"yihaoang/tsgbench","repo_kind":"official","path":"src/ts2vec.py","file_url":"https://github.com/yihaoang/tsgbench/blob/HEAD/src/ts2vec.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"acda587a1d1be3bf"}},{"code_sha256_prefix":"9f3f5233a9f7a99f","entry":"histogram_torch","repo":"yihaoang/tsgbench","repo_kind":"official","path":"src/feature_based_measures.py","file_url":"https://github.com/yihaoang/tsgbench/blob/HEAD/src/feature_based_measures.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9f3f5233a9f7a99f"}},{"code_sha256_prefix":"7b2be93e5d615cd6","entry":"sliding_window_view","repo":"yihaoang/tsgbench","repo_kind":"official","path":"src/preprocess.py","file_url":"https://github.com/yihaoang/tsgbench/blob/HEAD/src/preprocess.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7b2be93e5d615cd6"}},{"code_sha256_prefix":"4c67aa09d6ef1aed","entry":"train_test_divide","repo":"yihaoang/tsgbench","repo_kind":"official","path":"src/ds_ps.py","file_url":"https://github.com/yihaoang/tsgbench/blob/HEAD/src/ds_ps.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4c67aa09d6ef1aed"}},{"code_sha256_prefix":"c2d8cce2e180dada","entry":"discriminative_score_metrics","repo":"yihaoang/tsgbench","repo_kind":"official","path":"src/ds_ps.py","file_url":"https://github.com/yihaoang/tsgbench/blob/HEAD/src/ds_ps.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c2d8cce2e180dada"}},{"code_sha256_prefix":"98a90a05f7630bee","entry":"initialize_ts2vec","repo":"yihaoang/tsgbench","repo_kind":"official","path":"src/ts2vec.py","file_url":"https://github.com/yihaoang/tsgbench/blob/HEAD/src/ts2vec.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"98a90a05f7630bee"}},{"code_sha256_prefix":"c086625598b859b5","entry":"predictive_score_metrics","repo":"yihaoang/tsgbench","repo_kind":"official","path":"src/ds_ps.py","file_url":"https://github.com/yihaoang/tsgbench/blob/HEAD/src/ds_ps.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c086625598b859b5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}