{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-probabilistic-theory-of-deep-learning","title":"A Probabilistic Theory of Deep Learning","arxiv_id":"1504.00641","date":"2015-04-02","proceeding":null,"authors":["Ankit B. Patel","Tan Nguyen","Richard G. Baraniuk"],"abstract":"A grand challenge in machine learning is the development of computational\nalgorithms that match or outperform humans in perceptual inference tasks that\nare complicated by nuisance variation. For instance, visual object recognition\ninvolves the unknown object position, orientation, and scale in object\nrecognition while speech recognition involves the unknown voice pronunciation,\npitch, and speed. Recently, a new breed of deep learning algorithms have\nemerged for high-nuisance inference tasks that routinely yield pattern\nrecognition systems with near- or super-human capabilities. But a fundamental\nquestion remains: Why do they work? Intuitions abound, but a coherent framework\nfor understanding, analyzing, and synthesizing deep learning architectures has\nremained elusive. We answer this question by developing a new probabilistic\nframework for deep learning based on the Deep Rendering Model: a generative\nprobabilistic model that explicitly captures latent nuisance variation. By\nrelaxing the generative model to a discriminative one, we can recover two of\nthe current leading deep learning systems, deep convolutional neural networks\nand random decision forests, providing insights into their successes and\nshortcomings, as well as a principled route to their improvement.","url_abs":"http://arxiv.org/abs/1504.00641v1","url_pdf":"http://arxiv.org/pdf/1504.00641v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-probabilistic-theory-of-deep-learning","repo_url":"https://github.com/Kodiologist/Citematic","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-recognition","task_name":"Object Recognition"},{"task_slug":"speech-recognition","task_name":"Speech Recognition"},{"task_slug":"speech-recognition-1","task_name":"speech-recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1504.00641","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}