{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/accelerated-linearized-laplace-approximation","title":"Accelerated Linearized Laplace Approximation for Bayesian Deep Learning","arxiv_id":"2210.12642","date":"2022-10-23","proceeding":null,"authors":["Zhijie Deng","Feng Zhou","Jun Zhu"],"abstract":"Laplace approximation (LA) and its linearized variant (LLA) enable effortless adaptation of pretrained deep neural networks to Bayesian neural networks. The generalized Gauss-Newton (GGN) approximation is typically introduced to improve their tractability. However, LA and LLA are still confronted with non-trivial inefficiency issues and should rely on Kronecker-factored, diagonal, or even last-layer approximate GGN matrices in practical use. These approximations are likely to harm the fidelity of learning outcomes. To tackle this issue, inspired by the connections between LLA and neural tangent kernels (NTKs), we develop a Nystrom approximation to NTKs to accelerate LLA. Our method benefits from the capability of popular deep learning libraries for forward mode automatic differentiation, and enjoys reassuring theoretical guarantees. Extensive studies reflect the merits of the proposed method in aspects of both scalability and performance. Our method can even scale up to architectures like vision transformers. We also offer valuable ablation studies to diagnose our method. Code is available at \\url{https://github.com/thudzj/ELLA}.","url_abs":"https://arxiv.org/abs/2210.12642v1","url_pdf":"https://arxiv.org/pdf/2210.12642v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"accelerated-linearized-laplace-approximation","repo_url":"https://github.com/thudzj/ella","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2210.12642","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.12642"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/thudzj/ELLA","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thudzj/ella","reach":{"status":"ok"}}],"summary":{"ran":1,"ran_draft_wrong":3,"ran_honours":1,"unverified":3},"by_repo_kind":{"official":{"samples":8,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"fa3060dd8c01cfd0","entry":"FeatureExtractor","repo":"thudzj/ella","repo_kind":"official","path":"laplace/lllaplace.py","file_url":"https://github.com/thudzj/ella/blob/HEAD/laplace/lllaplace.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fa3060dd8c01cfd0"}},{"code_sha256_prefix":"b75b8c0c92ed9404","entry":"get_nll","repo":"thudzj/ella","repo_kind":"official","path":"laplace/lllaplace.py","file_url":"https://github.com/thudzj/ella/blob/HEAD/laplace/lllaplace.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b75b8c0c92ed9404"}},{"code_sha256_prefix":"fac13c13b577a2bd","entry":"normal_samples","repo":"thudzj/ella","repo_kind":"official","path":"laplace/lllaplace.py","file_url":"https://github.com/thudzj/ella/blob/HEAD/laplace/lllaplace.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fac13c13b577a2bd"}},{"code_sha256_prefix":"697b869e1f390443","entry":"parameters_per_layer","repo":"thudzj/ella","repo_kind":"official","path":"laplace/lllaplace.py","file_url":"https://github.com/thudzj/ella/blob/HEAD/laplace/lllaplace.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"697b869e1f390443"}},{"code_sha256_prefix":"3605582f0e4c4cbf","entry":"validate","repo":"thudzj/ella","repo_kind":"official","path":"laplace/lllaplace.py","file_url":"https://github.com/thudzj/ella/blob/HEAD/laplace/lllaplace.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3605582f0e4c4cbf"}},{"code_sha256_prefix":"437b96092eca33cb","entry":"BaseLaplace","repo":"thudzj/ella","repo_kind":"official","path":"laplace/lllaplace.py","file_url":"https://github.com/thudzj/ella/blob/HEAD/laplace/lllaplace.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"437b96092eca33cb"}},{"code_sha256_prefix":"67b95edf6bf0e55d","entry":"LLLaplace","repo":"thudzj/ella","repo_kind":"official","path":"laplace/lllaplace.py","file_url":"https://github.com/thudzj/ella/blob/HEAD/laplace/lllaplace.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"67b95edf6bf0e55d"}},{"code_sha256_prefix":"2ed85958abd08edd","entry":"ParametricLaplace","repo":"thudzj/ella","repo_kind":"official","path":"laplace/lllaplace.py","file_url":"https://github.com/thudzj/ella/blob/HEAD/laplace/lllaplace.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2ed85958abd08edd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}