{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/maximally-informative-hierarchical","title":"Maximally Informative Hierarchical Representations of High-Dimensional Data","arxiv_id":"1410.7404","date":"2014-10-27","proceeding":null,"authors":["Greg Ver Steeg","Aram Galstyan"],"abstract":"We consider a set of probabilistic functions of some input variables as a\nrepresentation of the inputs. We present bounds on how informative a\nrepresentation is about input data. We extend these bounds to hierarchical\nrepresentations so that we can quantify the contribution of each layer towards\ncapturing the information in the original data. The special form of these\nbounds leads to a simple, bottom-up optimization procedure to construct\nhierarchical representations that are also maximally informative about the\ndata. This optimization has linear computational complexity and constant sample\ncomplexity in the number of variables. These results establish a new approach\nto unsupervised learning of deep representations that is both principled and\npractical. We demonstrate the usefulness of the approach on both synthetic and\nreal-world data.","url_abs":"http://arxiv.org/abs/1410.7404v2","url_pdf":"http://arxiv.org/pdf/1410.7404v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"maximally-informative-hierarchical","repo_url":"https://github.com/gregversteeg/CorEx","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"GPL-2.0"}},{"paper_slug":"maximally-informative-hierarchical","repo_url":"https://github.com/gregversteeg/corex_topic","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"maximally-informative-hierarchical","repo_url":"https://github.com/gregversteeg/discrete_sieve","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"high","task_name":"Vocal Bursts Intensity Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1410.7404","atlas_url":"https://app.syntology.ai/?focus=1410.7404","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}