{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/toward-falsifying-causal-graphs-using-a","title":"Toward Falsifying Causal Graphs Using a Permutation-Based Test","arxiv_id":"2305.09565","date":"2023-05-16","proceeding":null,"authors":["Elias Eulig","Atalanti A. Mastakouri","Patrick Blöbaum","Michaela Hardt","Dominik Janzing"],"abstract":"Understanding causal relationships among the variables of a system is paramount to explain and control its behavior. For many real-world systems, however, the true causal graph is not readily available and one must resort to predictions made by algorithms or domain experts. Therefore, metrics that quantitatively assess the goodness of a causal graph provide helpful checks before using it in downstream tasks. Existing metrics provide an $\\textit{absolute}$ number of inconsistencies between the graph and the observed data, and without a baseline, practitioners are left to answer the hard question of how many such inconsistencies are acceptable or expected. Here, we propose a novel consistency metric by constructing a baseline through node permutations. By comparing the number of inconsistencies with those on the baseline, we derive an interpretable metric that captures whether the graph is significantly better than random. Evaluating on both simulated and real data sets from various domains, including biology and cloud monitoring, we demonstrate that the true graph is not falsified by our metric, whereas the wrong graphs given by a hypothetical user are likely to be falsified.","url_abs":"https://arxiv.org/abs/2305.09565v2","url_pdf":"https://arxiv.org/pdf/2305.09565v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"toward-falsifying-causal-graphs-using-a","repo_url":"https://github.com/eeulig/dag-falsification","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.09565","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.09565"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/eeulig/dag-falsification","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":5},"by_repo_kind":{"official":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e019074ea19a4bc8","entry":"domain_expert_nodes","repo":"eeulig/dag-falsification","repo_kind":"official","path":"falsifydags/utils/graph.py","file_url":"https://github.com/eeulig/dag-falsification/blob/HEAD/falsifydags/utils/graph.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e019074ea19a4bc8"}},{"code_sha256_prefix":"c38e86388263f32f","entry":"load_json","repo":"eeulig/dag-falsification","repo_kind":"official","path":"falsifydags/utils/auxiliaries.py","file_url":"https://github.com/eeulig/dag-falsification/blob/HEAD/falsifydags/utils/auxiliaries.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c38e86388263f32f"}},{"code_sha256_prefix":"90be22aede172f67","entry":"load_obj","repo":"eeulig/dag-falsification","repo_kind":"official","path":"falsifydags/utils/auxiliaries.py","file_url":"https://github.com/eeulig/dag-falsification/blob/HEAD/falsifydags/utils/auxiliaries.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"90be22aede172f67"}},{"code_sha256_prefix":"798a3eba82feced7","entry":"random_permute_adj","repo":"eeulig/dag-falsification","repo_kind":"official","path":"falsifydags/utils/graph.py","file_url":"https://github.com/eeulig/dag-falsification/blob/HEAD/falsifydags/utils/graph.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"798a3eba82feced7"}},{"code_sha256_prefix":"99bc8b43c3951099","entry":"simulate_dag_erdos_renyi","repo":"eeulig/dag-falsification","repo_kind":"official","path":"falsifydags/utils/graph.py","file_url":"https://github.com/eeulig/dag-falsification/blob/HEAD/falsifydags/utils/graph.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"99bc8b43c3951099"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}