{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-bayesian-alternative-to-mutual-information","title":"A Bayesian alternative to mutual information for the hierarchical clustering of dependent random variables","arxiv_id":"1501.05194","date":"2015-01-21","proceeding":null,"authors":["Guillaume Marrelec","Arnaud Messé","Pierre Bellec"],"abstract":"The use of mutual information as a similarity measure in agglomerative\nhierarchical clustering (AHC) raises an important issue: some correction needs\nto be applied for the dimensionality of variables. In this work, we formulate\nthe decision of merging dependent multivariate normal variables in an AHC\nprocedure as a Bayesian model comparison. We found that the Bayesian\nformulation naturally shrinks the empirical covariance matrix towards a matrix\nset a priori (e.g., the identity), provides an automated stopping rule, and\ncorrects for dimensionality using a term that scales up the measure as a\nfunction of the dimensionality of the variables. Also, the resulting log Bayes\nfactor is asymptotically proportional to the plug-in estimate of mutual\ninformation, with an additive correction for dimensionality in agreement with\nthe Bayesian information criterion. We investigated the behavior of these\nBayesian alternatives (in exact and asymptotic forms) to mutual information on\nsimulated and real data. An encouraging result was first derived on\nsimulations: the hierarchical clustering based on the log Bayes factor\noutperformed off-the-shelf clustering techniques as well as raw and normalized\nmutual information in terms of classification accuracy. On a toy example, we\nfound that the Bayesian approaches led to results that were similar to those of\nmutual information clustering techniques, with the advantage of an automated\nthresholding. On real functional magnetic resonance imaging (fMRI) datasets\nmeasuring brain activity, it identified clusters consistent with the\nestablished outcome of standard procedures. On this application, normalized\nmutual information had a highly atypical behavior, in the sense that it\nsystematically favored very large clusters. These initial experiments suggest\nthat the proposed Bayesian alternatives to mutual information are a useful new\ntool for hierarchical clustering.","url_abs":"http://arxiv.org/abs/1501.05194v2","url_pdf":"http://arxiv.org/pdf/1501.05194v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-bayesian-alternative-to-mutual-information","repo_url":"https://github.com/SIMEXP/arXiv-1501.05194","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"clustering","task_name":"Clustering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}