{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/data-generating-process-to-evaluate-causal","title":"Data Generating Process to Evaluate Causal Discovery Techniques for Time Series Data","arxiv_id":"2104.08043","date":"2021-04-16","proceeding":null,"authors":["Andrew R. Lawrence","Marcus Kaiser","Rui Sampaio","Maksim Sipos"],"abstract":"Going beyond correlations, the understanding and identification of causal relationships in observational time series, an important subfield of Causal Discovery, poses a major challenge. The lack of access to a well-defined ground truth for real-world data creates the need to rely on synthetic data for the evaluation of these methods. Existing benchmarks are limited in their scope, as they either are restricted to a \"static\" selection of data sets, or do not allow for a granular assessment of the methods' performance when commonly made assumptions are violated. We propose a flexible and simple to use framework for generating time series data, which is aimed at developing, evaluating, and benchmarking time series causal discovery methods. In particular, the framework can be used to fine tune novel methods on vast amounts of data, without \"overfitting\" them to a benchmark, but rather so they perform well in real-world use cases. Using our framework, we evaluate prominent time series causal discovery methods and demonstrate a notable degradation in performance when their assumptions are invalidated and their sensitivity to choice of hyperparameters. Finally, we propose future research directions and how our framework can support both researchers and practitioners.","url_abs":"https://arxiv.org/abs/2104.08043v1","url_pdf":"https://arxiv.org/pdf/2104.08043v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"data-generating-process-to-evaluate-causal","repo_url":"https://github.com/causalens/cdml-neurips2020","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"causal-discovery","task_name":"Causal Discovery"},{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series","task_name":"Time Series Analysis"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2104.08043","atlas_url":"https://app.syntology.ai/?focus=2104.08043","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2104.08043"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/causalens/cdml-neurips2020","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"14a6aee06ee02e2e","entry":"monotonic_function","repo":"causalens/cdml-neurips2020","repo_kind":"official","path":"dgp/src/structural_equations.py","file_url":"https://github.com/causalens/cdml-neurips2020/blob/HEAD/dgp/src/structural_equations.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"14a6aee06ee02e2e"}},{"code_sha256_prefix":"3f81ba838cb7f85e","entry":"piecewise_linear_function","repo":"causalens/cdml-neurips2020","repo_kind":"official","path":"dgp/src/structural_equations.py","file_url":"https://github.com/causalens/cdml-neurips2020/blob/HEAD/dgp/src/structural_equations.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"3f81ba838cb7f85e"}},{"code_sha256_prefix":"704144064385a4f0","entry":"trigonometric_function","repo":"causalens/cdml-neurips2020","repo_kind":"official","path":"dgp/src/structural_equations.py","file_url":"https://github.com/causalens/cdml-neurips2020/blob/HEAD/dgp/src/structural_equations.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"704144064385a4f0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}