{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-information-sieve","title":"The Information Sieve","arxiv_id":"1507.02284","date":"2015-07-08","proceeding":null,"authors":["Greg Ver Steeg","Aram Galstyan"],"abstract":"We introduce a new framework for unsupervised learning of representations\nbased on a novel hierarchical decomposition of information. Intuitively, data\nis passed through a series of progressively fine-grained sieves. Each layer of\nthe sieve recovers a single latent factor that is maximally informative about\nmultivariate dependence in the data. The data is transformed after each pass so\nthat the remaining unexplained information trickles down to the next layer.\nUltimately, we are left with a set of latent factors explaining all the\ndependence in the original data and remainder information consisting of\nindependent noise. We present a practical implementation of this framework for\ndiscrete variables and apply it to a variety of fundamental tasks in\nunsupervised learning including independent component analysis, lossy and\nlossless compression, and predicting missing values in data.","url_abs":"http://arxiv.org/abs/1507.02284v3","url_pdf":"http://arxiv.org/pdf/1507.02284v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-information-sieve","repo_url":"https://github.com/gregversteeg/discrete_sieve","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"the-information-sieve","repo_url":"https://github.com/gregversteeg/LinearSieve","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"missing-values","task_name":"Missing Values"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}