{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/applicability-of-random-matrix-theory-in-deep","title":"Appearance of Random Matrix Theory in Deep Learning","arxiv_id":"2102.06740","date":"2021-02-12","proceeding":null,"authors":["Nicholas P Baskerville","Diego Granziol","Jonathan P Keating"],"abstract":"We investigate the local spectral statistics of the loss surface Hessians of artificial neural networks, where we discover excellent agreement with Gaussian Orthogonal Ensemble statistics across several network architectures and datasets. These results shed new light on the applicability of Random Matrix Theory to modelling neural networks and suggest a previously unrecognised role for it in the study of loss surfaces in deep learning. Inspired by these observations, we propose a novel model for the true loss surfaces of neural networks, consistent with our observations, which allows for Hessian spectral densities with rank degeneracy and outliers, extensively observed in practice, and predicts a growing independence of loss gradients as a function of distance in weight-space. We further investigate the importance of the true loss surface in neural networks and find, in contrast to previous work, that the exponential hardness of locating the global minimum has practical consequences for achieving state of the art performance.","url_abs":"https://arxiv.org/abs/2102.06740v3","url_pdf":"https://arxiv.org/pdf/2102.06740v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"applicability-of-random-matrix-theory-in-deep","repo_url":"https://github.com/npbaskerville/dnn-rmt-spacings","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2102.06740","atlas_url":"https://app.syntology.ai/?focus=2102.06740","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2102.06740"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/npbaskerville/dnn-rmt-spacings","reach":null}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"ea920a1aa1edfb66","entry":"new_schedule","repo":"npbaskerville/dnn-rmt-spacings","repo_kind":"official","path":"GadamX/run_sgd.py","file_url":"https://github.com/npbaskerville/dnn-rmt-spacings/blob/HEAD/GadamX/run_sgd.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"ea920a1aa1edfb66"}},{"code_sha256_prefix":"88db7bf86da5bc92","entry":"schedule","repo":"npbaskerville/dnn-rmt-spacings","repo_kind":"official","path":"GadamX/run_sgd.py","file_url":"https://github.com/npbaskerville/dnn-rmt-spacings/blob/HEAD/GadamX/run_sgd.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"88db7bf86da5bc92"}},{"code_sha256_prefix":"3c266bea1aedc7d1","entry":"schedule_piecewise_const","repo":"npbaskerville/dnn-rmt-spacings","repo_kind":"official","path":"GadamX/run_sgd.py","file_url":"https://github.com/npbaskerville/dnn-rmt-spacings/blob/HEAD/GadamX/run_sgd.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"3c266bea1aedc7d1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}