{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/probabilistic-backpropagation-for-scalable","title":"Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks","arxiv_id":"1502.05336","date":"2015-02-18","proceeding":null,"authors":["José Miguel Hernández-Lobato","Ryan P. Adams"],"abstract":"Large multilayer neural networks trained with backpropagation have recently\nachieved state-of-the-art results in a wide range of problems. However, using\nbackprop for neural net learning still has some disadvantages, e.g., having to\ntune a large number of hyperparameters to the data, lack of calibrated\nprobabilistic predictions, and a tendency to overfit the training data. In\nprinciple, the Bayesian approach to learning neural networks does not have\nthese problems. However, existing Bayesian techniques lack scalability to large\ndataset and network sizes. In this work we present a novel scalable method for\nlearning Bayesian neural networks, called probabilistic backpropagation (PBP).\nSimilar to classical backpropagation, PBP works by computing a forward\npropagation of probabilities through the network and then doing a backward\ncomputation of gradients. A series of experiments on ten real-world datasets\nshow that PBP is significantly faster than other techniques, while offering\ncompetitive predictive abilities. Our experiments also show that PBP provides\naccurate estimates of the posterior variance on the network weights.","url_abs":"http://arxiv.org/abs/1502.05336v2","url_pdf":"http://arxiv.org/pdf/1502.05336v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"probabilistic-backpropagation-for-scalable","repo_url":"https://github.com/JoonHyung-Park/DeepEnsemble","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"probabilistic-backpropagation-for-scalable","repo_url":"https://github.com/ThirstyScholar/bayes-by-backprop","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"probabilistic-backpropagation-for-scalable","repo_url":"https://github.com/xuangu-fang/Streaming-Bayesian-Deep-Tensor","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1502.05336","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}