{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/statistical-mechanics-of-deep-learning-beyond","title":"A statistical mechanics framework for Bayesian deep neural networks beyond the infinite-width limit","arxiv_id":"2209.04882","date":"2022-09-11","proceeding":null,"authors":["R. Pacelli","S. Ariosto","M. Pastore","F. Ginelli","M. Gherardi","P. Rotondo"],"abstract":"Despite the practical success of deep neural networks, a comprehensive theoretical framework that can predict practically relevant scores, such as the test accuracy, from knowledge of the training data is currently lacking. Huge simplifications arise in the infinite-width limit, where the number of units $N_\\ell$ in each hidden layer ($\\ell=1,\\dots, L$, being $L$ the depth of the network) far exceeds the number $P$ of training examples. This idealisation, however, blatantly departs from the reality of deep learning practice. Here, we use the toolset of statistical mechanics to overcome these limitations and derive an approximate partition function for fully-connected deep neural architectures, which encodes information about the trained models. The computation holds in the ''thermodynamic limit'' where both $N_\\ell$ and $P$ are large and their ratio $\\alpha_\\ell = P/N_\\ell$ is finite. This advance allows us to obtain (i) a closed formula for the generalisation error associated to a regression task in a one-hidden layer network with finite $\\alpha_1$; (ii) an approximate expression of the partition function for deep architectures (via an ''effective action'' that depends on a finite number of ''order parameters''); (iii) a link between deep neural networks in the proportional asymptotic limit and Student's $t$ processes.","url_abs":"https://arxiv.org/abs/2209.04882v5","url_pdf":"https://arxiv.org/pdf/2209.04882v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"statistical-mechanics-of-deep-learning-beyond","repo_url":"https://github.com/rpacelli/fc_deep_bayesian_networks","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Unlicense"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2209.04882","atlas_url":"https://app.syntology.ai/?focus=2209.04882","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2209.04882"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rpacelli/fc_deep_bayesian_networks","reach":{"status":"ok","spdx":"Unlicense"}}],"summary":{"ran":8},"by_repo_kind":{"official":{"samples":8,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"67c39864469f28c6","entry":"CorrMat","repo":"rpacelli/fc_deep_bayesian_networks","repo_kind":"official","path":"theory.py","file_url":"https://github.com/rpacelli/fc_deep_bayesian_networks/blob/HEAD/theory.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Unlicense","inline_ok":true,"mcp_get_code":{"code_sha256":"67c39864469f28c6"}},{"code_sha256_prefix":"e633d84143b2409c","entry":"check_file","repo":"rpacelli/fc_deep_bayesian_networks","repo_kind":"official","path":"data_analysis/data_analysis.py","file_url":"https://github.com/rpacelli/fc_deep_bayesian_networks/blob/HEAD/data_analysis/data_analysis.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Unlicense","inline_ok":true,"mcp_get_code":{"code_sha256":"e633d84143b2409c"}},{"code_sha256_prefix":"31c107ff46c82600","entry":"find_device","repo":"rpacelli/fc_deep_bayesian_networks","repo_kind":"official","path":"utils.py","file_url":"https://github.com/rpacelli/fc_deep_bayesian_networks/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Unlicense","inline_ok":true,"mcp_get_code":{"code_sha256":"31c107ff46c82600"}},{"code_sha256_prefix":"8292db5fe4044158","entry":"k0","repo":"rpacelli/fc_deep_bayesian_networks","repo_kind":"official","path":"theory.py","file_url":"https://github.com/rpacelli/fc_deep_bayesian_networks/blob/HEAD/theory.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Unlicense","inline_ok":true,"mcp_get_code":{"code_sha256":"8292db5fe4044158"}},{"code_sha256_prefix":"0e71a1474f7716e0","entry":"kmatrix","repo":"rpacelli/fc_deep_bayesian_networks","repo_kind":"official","path":"theory.py","file_url":"https://github.com/rpacelli/fc_deep_bayesian_networks/blob/HEAD/theory.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Unlicense","inline_ok":true,"mcp_get_code":{"code_sha256":"0e71a1474f7716e0"}},{"code_sha256_prefix":"9b91ed66d4895edf","entry":"load_run","repo":"rpacelli/fc_deep_bayesian_networks","repo_kind":"official","path":"data_analysis/data_analysis.py","file_url":"https://github.com/rpacelli/fc_deep_bayesian_networks/blob/HEAD/data_analysis/data_analysis.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Unlicense","inline_ok":true,"mcp_get_code":{"code_sha256":"9b91ed66d4895edf"}},{"code_sha256_prefix":"76930061f3339640","entry":"test","repo":"rpacelli/fc_deep_bayesian_networks","repo_kind":"official","path":"utils.py","file_url":"https://github.com/rpacelli/fc_deep_bayesian_networks/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Unlicense","inline_ok":true,"mcp_get_code":{"code_sha256":"76930061f3339640"}},{"code_sha256_prefix":"3a5c5b3ec170bafa","entry":"train","repo":"rpacelli/fc_deep_bayesian_networks","repo_kind":"official","path":"utils.py","file_url":"https://github.com/rpacelli/fc_deep_bayesian_networks/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Unlicense","inline_ok":true,"mcp_get_code":{"code_sha256":"3a5c5b3ec170bafa"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}