{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bayesian-layers-a-module-for-neural-network","title":"Bayesian Layers: A Module for Neural Network Uncertainty","arxiv_id":"1812.03973","date":"2018-12-10","proceeding":"NeurIPS 2019 12","authors":["Dustin Tran","Michael W. Dusenberry","Mark van der Wilk","Danijar Hafner"],"abstract":"We describe Bayesian Layers, a module designed for fast experimentation with\nneural network uncertainty. It extends neural network libraries with drop-in\nreplacements for common layers. This enables composition via a unified\nabstraction over deterministic and stochastic functions and allows for\nscalability via the underlying system. These layers capture uncertainty over\nweights (Bayesian neural nets), pre-activation units (dropout), activations\n(\"stochastic output layers\"), or the function itself (Gaussian processes). They\ncan also be reversible to propagate uncertainty from input to output. We\ninclude code examples for common architectures such as Bayesian LSTMs, deep\nGPs, and flow-based models. As demonstration, we fit a 5-billion parameter\n\"Bayesian Transformer\" on 512 TPUv2 cores for uncertainty in machine\ntranslation and a Bayesian dynamics model for model-based planning. Finally, we\nshow how Bayesian Layers can be used within the Edward2 probabilistic\nprogramming language for probabilistic programs with stochastic processes.","url_abs":"http://arxiv.org/abs/1812.03973v3","url_pdf":"http://arxiv.org/pdf/1812.03973v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bayesian-layers-a-module-for-neural-network","repo_url":"https://github.com/google/edward2","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"gaussian-processes","task_name":"Gaussian Processes"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"probabilistic-programming","task_name":"Probabilistic Programming"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1812.03973","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}