{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-simple-baseline-for-bayesian-uncertainty-in","title":"A Simple Baseline for Bayesian Uncertainty in Deep Learning","arxiv_id":"1902.02476","date":"2019-02-07","proceeding":"NeurIPS 2019 12","authors":["Wesley Maddox","Timur Garipov","Pavel Izmailov","Dmitry Vetrov","Andrew Gordon Wilson"],"abstract":"We propose SWA-Gaussian (SWAG), a simple, scalable, and general purpose approach for uncertainty representation and calibration in deep learning. Stochastic Weight Averaging (SWA), which computes the first moment of stochastic gradient descent (SGD) iterates with a modified learning rate schedule, has recently been shown to improve generalization in deep learning. With SWAG, we fit a Gaussian using the SWA solution as the first moment and a low rank plus diagonal covariance also derived from the SGD iterates, forming an approximate posterior distribution over neural network weights; we then sample from this Gaussian distribution to perform Bayesian model averaging. We empirically find that SWAG approximates the shape of the true posterior, in accordance with results describing the stationary distribution of SGD iterates. Moreover, we demonstrate that SWAG performs well on a wide variety of tasks, including out of sample detection, calibration, and transfer learning, in comparison to many popular alternatives including MC dropout, KFAC Laplace, SGLD, and temperature scaling.","url_abs":"https://arxiv.org/abs/1902.02476v2","url_pdf":"https://arxiv.org/pdf/1902.02476v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-simple-baseline-for-bayesian-uncertainty-in","repo_url":"https://github.com/wjmaddox/swa_gaussian","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-2-Clause"}},{"paper_slug":"a-simple-baseline-for-bayesian-uncertainty-in","repo_url":"https://github.com/NajibYavari/DD2412","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"a-simple-baseline-for-bayesian-uncertainty-in","repo_url":"https://github.com/SamuelGuilluy/Bayesian_ML_SWAG","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"a-simple-baseline-for-bayesian-uncertainty-in","repo_url":"https://github.com/YeongHyeon/SWA-Gaussian-TF2","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"a-simple-baseline-for-bayesian-uncertainty-in","repo_url":"https://github.com/abdulmajid-murad/deep_probabilistic_forecast","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"a-simple-baseline-for-bayesian-uncertainty-in","repo_url":"https://github.com/lamantinushka/StructuredCovariance","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"a-simple-baseline-for-bayesian-uncertainty-in","repo_url":"https://github.com/ENSTA-U2IS-AI/torch-uncertainty","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"a-simple-baseline-for-bayesian-uncertainty-in","repo_url":"https://github.com/lightning-uq-box/lightning-uq-box","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"bayesian-inference","task_name":"Bayesian Inference"},{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"uncertainty-quantification","task_name":"Uncertainty Quantification"}],"methods":[{"method_slug":"sgd","method_name":"SGD"},{"method_slug":"stochastic-weight-averaging","method_name":"Stochastic Weight Averaging"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1902.02476","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1902.02476"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ENSTA-U2IS-AI/torch-uncertainty","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wjmaddox/swa_gaussian","reach":{"status":"ok","spdx":"BSD-2-Clause"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lightning-uq-box/lightning-uq-box","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SamuelGuilluy/Bayesian_ML_SWAG","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/abdulmajid-murad/deep_probabilistic_forecast","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/NajibYavari/DD2412","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/YeongHyeon/SWA-Gaussian-TF2","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lamantinushka/StructuredCovariance","reach":{"status":"ok"}}],"summary":{"ran":5,"ran_draft_wrong":1,"unverified":11},"by_repo_kind":{"official":{"samples":14,"ran":6,"repositories":1},"listed":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e11996f54eea8253","entry":"adversarial_cross_entropy","repo":"wjmaddox/swa_gaussian","repo_kind":"official","path":"swag/losses.py","file_url":"https://github.com/wjmaddox/swa_gaussian/blob/HEAD/swag/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"e11996f54eea8253"}},{"code_sha256_prefix":"9d5555c9ff725b7f","entry":"center_crop","repo":"wjmaddox/swa_gaussian","repo_kind":"official","path":"swag/models/layers.py","file_url":"https://github.com/wjmaddox/swa_gaussian/blob/HEAD/swag/models/layers.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"9d5555c9ff725b7f"}},{"code_sha256_prefix":"583f9780bdd00a45","entry":"conv3x3","repo":"wjmaddox/swa_gaussian","repo_kind":"official","path":"swag/models/preresnet.py","file_url":"https://github.com/wjmaddox/swa_gaussian/blob/HEAD/swag/models/preresnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"583f9780bdd00a45"}},{"code_sha256_prefix":"fc2aaf6fd20bc53f","entry":"cross_entropy","repo":"wjmaddox/swa_gaussian","repo_kind":"official","path":"swag/losses.py","file_url":"https://github.com/wjmaddox/swa_gaussian/blob/HEAD/swag/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"fc2aaf6fd20bc53f"}},{"code_sha256_prefix":"1d7c68b369e70ab0","entry":"has_file_allowed_extension","repo":"wjmaddox/swa_gaussian","repo_kind":"official","path":"swag/camvid.py","file_url":"https://github.com/wjmaddox/swa_gaussian/blob/HEAD/swag/camvid.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"1d7c68b369e70ab0"}},{"code_sha256_prefix":"af447e65a04b459e","entry":"is_image_file","repo":"wjmaddox/swa_gaussian","repo_kind":"official","path":"swag/camvid.py","file_url":"https://github.com/wjmaddox/swa_gaussian/blob/HEAD/swag/camvid.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"af447e65a04b459e"}},{"code_sha256_prefix":"b99c1d7b48976895","entry":"bayesian_sampling","repo":"YeongHyeon/SWA-Gaussian-TF2","repo_kind":"listed","path":"source/utils.py","file_url":"https://github.com/YeongHyeon/SWA-Gaussian-TF2/blob/HEAD/source/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b99c1d7b48976895"}},{"code_sha256_prefix":"e1ae5ee1960f180e","entry":"camvid_loaders","repo":"wjmaddox/swa_gaussian","repo_kind":"official","path":"swag/data.py","file_url":"https://github.com/wjmaddox/swa_gaussian/blob/HEAD/swag/data.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"e1ae5ee1960f180e"}},{"code_sha256_prefix":"28c6926bada96fe8","entry":"find_models","repo":"wjmaddox/swa_gaussian","repo_kind":"official","path":"experiments/ensembling/swa_ecdf_ensembles.py","file_url":"https://github.com/wjmaddox/swa_gaussian/blob/HEAD/experiments/ensembling/swa_ecdf_ensembles.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"28c6926bada96fe8"}},{"code_sha256_prefix":"99e3815602045644","entry":"loaders","repo":"wjmaddox/swa_gaussian","repo_kind":"official","path":"experiments/imagenet/data.py","file_url":"https://github.com/wjmaddox/swa_gaussian/blob/HEAD/experiments/imagenet/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"99e3815602045644"}},{"code_sha256_prefix":"9dc1a3ebe60f97ec","entry":"loaders","repo":"wjmaddox/swa_gaussian","repo_kind":"official","path":"swag/data.py","file_url":"https://github.com/wjmaddox/swa_gaussian/blob/HEAD/swag/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"9dc1a3ebe60f97ec"}},{"code_sha256_prefix":"8bbca7799326d580","entry":"masked_loss","repo":"wjmaddox/swa_gaussian","repo_kind":"official","path":"swag/losses.py","file_url":"https://github.com/wjmaddox/swa_gaussian/blob/HEAD/swag/losses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"8bbca7799326d580"}},{"code_sha256_prefix":"bbf4ede63ece446f","entry":"min_max_norm","repo":"YeongHyeon/SWA-Gaussian-TF2","repo_kind":"listed","path":"source/utils.py","file_url":"https://github.com/YeongHyeon/SWA-Gaussian-TF2/blob/HEAD/source/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bbf4ede63ece446f"}},{"code_sha256_prefix":"3ccff5ae337f212a","entry":"nll","repo":"wjmaddox/swa_gaussian","repo_kind":"official","path":"experiments/ensembling/swa_ecdf_ensembles.py","file_url":"https://github.com/wjmaddox/swa_gaussian/blob/HEAD/experiments/ensembling/swa_ecdf_ensembles.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"3ccff5ae337f212a"}},{"code_sha256_prefix":"7478bf872b01d269","entry":"schedule","repo":"wjmaddox/swa_gaussian","repo_kind":"official","path":"experiments/imagenet/run_swag_imagenet.py","file_url":"https://github.com/wjmaddox/swa_gaussian/blob/HEAD/experiments/imagenet/run_swag_imagenet.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"7478bf872b01d269"}},{"code_sha256_prefix":"d68d9e1382bea360","entry":"sorted_list","repo":"YeongHyeon/SWA-Gaussian-TF2","repo_kind":"listed","path":"source/utils.py","file_url":"https://github.com/YeongHyeon/SWA-Gaussian-TF2/blob/HEAD/source/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d68d9e1382bea360"}},{"code_sha256_prefix":"d1432df89765a54c","entry":"svhn_loaders","repo":"wjmaddox/swa_gaussian","repo_kind":"official","path":"swag/data.py","file_url":"https://github.com/wjmaddox/swa_gaussian/blob/HEAD/swag/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"d1432df89765a54c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}