{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-learning-is-singular-and-that-s-good-1","title":"Deep Learning is Singular, and That's Good","arxiv_id":"2010.11560","date":"2020-10-22","proceeding":null,"authors":["Daniel Murfet","Susan Wei","Mingming Gong","Hui Li","Jesse Gell-Redman","Thomas Quella"],"abstract":"In singular models, the optimal set of parameters forms an analytic set with singularities and classical statistical inference cannot be applied to such models. This is significant for deep learning as neural networks are singular and thus \"dividing\" by the determinant of the Hessian or employing the Laplace approximation are not appropriate. Despite its potential for addressing fundamental issues in deep learning, singular learning theory appears to have made little inroads into the developing canon of deep learning theory. Via a mix of theory and experiment, we present an invitation to singular learning theory as a vehicle for understanding deep learning and suggest important future work to make singular learning theory directly applicable to how deep learning is performed in practice.","url_abs":"https://arxiv.org/abs/2010.11560v1","url_pdf":"https://arxiv.org/pdf/2010.11560v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-learning-is-singular-and-that-s-good-1","repo_url":"https://github.com/susanwe/RLCT","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"learning-theory","task_name":"Learning Theory"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2010.11560","atlas_url":"https://app.syntology.ai/?focus=2010.11560","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2010.11560"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/susanwe/RLCT","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":1,"unverified":16},"by_repo_kind":{"official":{"samples":17,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f6b944f50d3f15ae","entry":"count_parameters","repo":"susanwe/RLCT","repo_kind":"official","path":"RLCT_helper.py","file_url":"https://github.com/susanwe/RLCT/blob/HEAD/RLCT_helper.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f6b944f50d3f15ae"}},{"code_sha256_prefix":"057d9f485a882881","entry":"approxinf_nll_implicit","repo":"susanwe/RLCT","repo_kind":"official","path":"implicit_vi.py","file_url":"https://github.com/susanwe/RLCT/blob/HEAD/implicit_vi.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"057d9f485a882881"}},{"code_sha256_prefix":"d5e823409fd99f99","entry":"conditioned_pyro_tanh","repo":"susanwe/RLCT","repo_kind":"official","path":"models.py","file_url":"https://github.com/susanwe/RLCT/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d5e823409fd99f99"}},{"code_sha256_prefix":"dd81cf51ce199054","entry":"expected_nll_posterior_relu","repo":"susanwe/RLCT","repo_kind":"official","path":"mcmc_helper.py","file_url":"https://github.com/susanwe/RLCT/blob/HEAD/mcmc_helper.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dd81cf51ce199054"}},{"code_sha256_prefix":"33119372a53640d5","entry":"expected_nll_posterior_tanh","repo":"susanwe/RLCT","repo_kind":"official","path":"mcmc_helper.py","file_url":"https://github.com/susanwe/RLCT/blob/HEAD/mcmc_helper.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"33119372a53640d5"}},{"code_sha256_prefix":"6ad62680134e71a5","entry":"get_data","repo":"susanwe/RLCT","repo_kind":"official","path":"langevin_monte_carlo.py","file_url":"https://github.com/susanwe/RLCT/blob/HEAD/langevin_monte_carlo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6ad62680134e71a5"}},{"code_sha256_prefix":"c669f2e8fa1858ea","entry":"get_data_symmetric","repo":"susanwe/RLCT","repo_kind":"official","path":"pyro_example.py","file_url":"https://github.com/susanwe/RLCT/blob/HEAD/pyro_example.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c669f2e8fa1858ea"}},{"code_sha256_prefix":"b698e60b64037283","entry":"get_dataset_by_id","repo":"susanwe/RLCT","repo_kind":"official","path":"dataset_factory.py","file_url":"https://github.com/susanwe/RLCT/blob/HEAD/dataset_factory.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b698e60b64037283"}},{"code_sha256_prefix":"5dc12c699e81fa9f","entry":"lsfit_lambda","repo":"susanwe/RLCT","repo_kind":"official","path":"RLCT_helper.py","file_url":"https://github.com/susanwe/RLCT/blob/HEAD/RLCT_helper.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5dc12c699e81fa9f"}},{"code_sha256_prefix":"fac793eb17aba0d5","entry":"nonlin","repo":"susanwe/RLCT","repo_kind":"official","path":"pyro_example.py","file_url":"https://github.com/susanwe/RLCT/blob/HEAD/pyro_example.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fac793eb17aba0d5"}},{"code_sha256_prefix":"57da358ba6236f52","entry":"plot_energy","repo":"susanwe/RLCT","repo_kind":"official","path":"langevin_monte_carlo.py","file_url":"https://github.com/susanwe/RLCT/blob/HEAD/langevin_monte_carlo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"57da358ba6236f52"}},{"code_sha256_prefix":"66467be486ae22c3","entry":"pyro_rr","repo":"susanwe/RLCT","repo_kind":"official","path":"models.py","file_url":"https://github.com/susanwe/RLCT/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"66467be486ae22c3"}},{"code_sha256_prefix":"a6730396985a56d1","entry":"pyro_tanh","repo":"susanwe/RLCT","repo_kind":"official","path":"models.py","file_url":"https://github.com/susanwe/RLCT/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a6730396985a56d1"}},{"code_sha256_prefix":"ca8547ea3ea72bef","entry":"run_inference","repo":"susanwe/RLCT","repo_kind":"official","path":"pyro_example.py","file_url":"https://github.com/susanwe/RLCT/blob/HEAD/pyro_example.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ca8547ea3ea72bef"}},{"code_sha256_prefix":"18d1e4cee617e2f6","entry":"sample_EVI","repo":"susanwe/RLCT","repo_kind":"official","path":"explicit_vi.py","file_url":"https://github.com/susanwe/RLCT/blob/HEAD/explicit_vi.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"18d1e4cee617e2f6"}},{"code_sha256_prefix":"d386ff8d9a8246b5","entry":"sample_IVI","repo":"susanwe/RLCT","repo_kind":"official","path":"implicit_vi.py","file_url":"https://github.com/susanwe/RLCT/blob/HEAD/implicit_vi.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d386ff8d9a8246b5"}},{"code_sha256_prefix":"2433cc567caefb30","entry":"theoretical_RLCT","repo":"susanwe/RLCT","repo_kind":"official","path":"RLCT_helper.py","file_url":"https://github.com/susanwe/RLCT/blob/HEAD/RLCT_helper.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2433cc567caefb30"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}