{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/entropic-causal-inference","title":"Entropic Causal Inference","arxiv_id":"1611.04035","date":"2016-11-12","proceeding":null,"authors":["Murat Kocaoglu","Alexandros G. Dimakis","Sriram Vishwanath","Babak Hassibi"],"abstract":"We consider the problem of identifying the causal direction between two\ndiscrete random variables using observational data. Unlike previous work, we\nkeep the most general functional model but make an assumption on the unobserved\nexogenous variable: Inspired by Occam's razor, we assume that the exogenous\nvariable is simple in the true causal direction. We quantify simplicity using\nR\\'enyi entropy. Our main result is that, under natural assumptions, if the\nexogenous variable has low $H_0$ entropy (cardinality) in the true direction,\nit must have high $H_0$ entropy in the wrong direction. We establish several\nalgorithmic hardness results about estimating the minimum entropy exogenous\nvariable. We show that the problem of finding the exogenous variable with\nminimum entropy is equivalent to the problem of finding minimum joint entropy\ngiven $n$ marginal distributions, also known as minimum entropy coupling\nproblem. We propose an efficient greedy algorithm for the minimum entropy\ncoupling problem, that for $n=2$ provably finds a local optimum. This gives a\ngreedy algorithm for finding the exogenous variable with minimum $H_1$ (Shannon\nEntropy). Our greedy entropy-based causal inference algorithm has similar\nperformance to the state of the art additive noise models in real datasets. One\nadvantage of our approach is that we make no use of the values of random\nvariables but only their distributions. Our method can therefore be used for\ncausal inference for both ordinal and also categorical data, unlike additive\nnoise models.","url_abs":"http://arxiv.org/abs/1611.04035v2","url_pdf":"http://arxiv.org/pdf/1611.04035v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"entropic-causal-inference","repo_url":"https://github.com/mkocaoglu/Entropic-Causality","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"causal-inference","task_name":"Causal Inference"}],"methods":[{"method_slug":"causal-inference","method_name":"Causal inference"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1611.04035","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}