{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/discovering-hidden-factors-of-variation-in","title":"Discovering Hidden Factors of Variation in Deep Networks","arxiv_id":"1412.6583","date":"2014-12-20","proceeding":null,"authors":["Brian Cheung","Jesse A. Livezey","Arjun K. Bansal","Bruno A. Olshausen"],"abstract":"Deep learning has enjoyed a great deal of success because of its ability to\nlearn useful features for tasks such as classification. But there has been less\nexploration in learning the factors of variation apart from the classification\nsignal. By augmenting autoencoders with simple regularization terms during\ntraining, we demonstrate that standard deep architectures can discover and\nexplicitly represent factors of variation beyond those relevant for\ncategorization. We introduce a cross-covariance penalty (XCov) as a method to\ndisentangle factors like handwriting style for digits and subject identity in\nfaces. We demonstrate this on the MNIST handwritten digit database, the Toronto\nFaces Database (TFD) and the Multi-PIE dataset by generating manipulated\ninstances of the data. Furthermore, we demonstrate these deep networks can\nextrapolate `hidden' variation in the supervised signal.","url_abs":"http://arxiv.org/abs/1412.6583v4","url_pdf":"http://arxiv.org/pdf/1412.6583v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"discovering-hidden-factors-of-variation-in","repo_url":"https://github.com/zjsong/CDNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"classification","task_name":"General Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1412.6583","atlas_url":"https://app.syntology.ai/?focus=1412.6583","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}