{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/variational-inference-for-infinitely-deep","title":"Variational Inference for Infinitely Deep Neural Networks","arxiv_id":"2209.10091","date":"2022-09-21","proceeding":null,"authors":["Achille Nazaret","David Blei"],"abstract":"We introduce the unbounded depth neural network (UDN), an infinitely deep probabilistic model that adapts its complexity to the training data. The UDN contains an infinite sequence of hidden layers and places an unbounded prior on a truncation L, the layer from which it produces its data. Given a dataset of observations, the posterior UDN provides a conditional distribution of both the parameters of the infinite neural network and its truncation. We develop a novel variational inference algorithm to approximate this posterior, optimizing a distribution of the neural network weights and of the truncation depth L, and without any upper limit on L. To this end, the variational family has a special structure: it models neural network weights of arbitrary depth, and it dynamically creates or removes free variational parameters as its distribution of the truncation is optimized. (Unlike heuristic approaches to model search, it is solely through gradient-based optimization that this algorithm explores the space of truncations.) We study the UDN on real and synthetic data. We find that the UDN adapts its posterior depth to the dataset complexity; it outperforms standard neural networks of similar computational complexity; and it outperforms other approaches to infinite-depth neural networks.","url_abs":"https://arxiv.org/abs/2209.10091v1","url_pdf":"https://arxiv.org/pdf/2209.10091v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"variational-inference-for-infinitely-deep","repo_url":"https://github.com/anazaret/unbounded-depth-neural-networks","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[{"method_slug":"variational-inference","method_name":"Variational Inference"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2209.10091","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2209.10091"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/anazaret/unbounded-depth-neural-networks","reach":null}],"summary":{"ran":3,"ran_draft_wrong":2,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"0d22c5d4130c0898","entry":"CategoricalDUN","repo":"anazaret/unbounded-depth-neural-networks","repo_kind":"official","path":"src/models.py","file_url":"https://github.com/anazaret/unbounded-depth-neural-networks/blob/HEAD/src/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0d22c5d4130c0898"}},{"code_sha256_prefix":"472b18a5cf0da465","entry":"FixedDepth","repo":"anazaret/unbounded-depth-neural-networks","repo_kind":"official","path":"src/models.py","file_url":"https://github.com/anazaret/unbounded-depth-neural-networks/blob/HEAD/src/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"472b18a5cf0da465"}},{"code_sha256_prefix":"72fccbd8711c422f","entry":"TruncatedPoisson","repo":"anazaret/unbounded-depth-neural-networks","repo_kind":"official","path":"src/models.py","file_url":"https://github.com/anazaret/unbounded-depth-neural-networks/blob/HEAD/src/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"72fccbd8711c422f"}},{"code_sha256_prefix":"2741eba57a8334ef","entry":"softplus_inverse","repo":"anazaret/unbounded-depth-neural-networks","repo_kind":"official","path":"src/models.py","file_url":"https://github.com/anazaret/unbounded-depth-neural-networks/blob/HEAD/src/models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2741eba57a8334ef"}},{"code_sha256_prefix":"e8c84302100af2af","entry":"softplus_inverse","repo":"anazaret/unbounded-depth-neural-networks","repo_kind":"official","path":"src/models.py","file_url":"https://github.com/anazaret/unbounded-depth-neural-networks/blob/HEAD/src/models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e8c84302100af2af"}},{"code_sha256_prefix":"817bf6df0c3a6e2a","entry":"UnboundedDepthNetwork","repo":"anazaret/unbounded-depth-neural-networks","repo_kind":"official","path":"src/models.py","file_url":"https://github.com/anazaret/unbounded-depth-neural-networks/blob/HEAD/src/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"817bf6df0c3a6e2a"}},{"code_sha256_prefix":"9c5bb28741680fc7","entry":"VariationalDepth","repo":"anazaret/unbounded-depth-neural-networks","repo_kind":"official","path":"src/models.py","file_url":"https://github.com/anazaret/unbounded-depth-neural-networks/blob/HEAD/src/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9c5bb28741680fc7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}