{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/traces-of-class-cross-class-structure-pervade","title":"Traces of Class/Cross-Class Structure Pervade Deep Learning Spectra","arxiv_id":"2008.11865","date":"2020-08-27","proceeding":null,"authors":["Vardan Papyan"],"abstract":"Numerous researchers recently applied empirical spectral analysis to the study of modern deep learning classifiers. We identify and discuss an important formal class/cross-class structure and show how it lies at the origin of the many visually striking features observed in deepnet spectra, some of which were reported in recent articles, others are unveiled here for the first time. These include spectral outliers, \"spikes\", and small but distinct continuous distributions, \"bumps\", often seen beyond the edge of a \"main bulk\". The significance of the cross-class structure is illustrated in three ways: (i) we prove the ratio of outliers to bulk in the spectrum of the Fisher information matrix is predictive of misclassification, in the context of multinomial logistic regression; (ii) we demonstrate how, gradually with depth, a network is able to separate class-distinctive information from class variability, all while orthogonalizing the class-distinctive information; and (iii) we propose a correction to KFAC, a well-known second-order optimization algorithm for training deepnets.","url_abs":"https://arxiv.org/abs/2008.11865v1","url_pdf":"https://arxiv.org/pdf/2008.11865v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"traces-of-class-cross-class-structure-pervade","repo_url":"https://github.com/f-dangel/curvlinops","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"articles","task_name":"Articles"},{"task_slug":"deep-learning","task_name":"Deep Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2008.11865","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2008.11865"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/f-dangel/curvlinops","reach":null}],"summary":{"ran_honours":2},"by_repo_kind":{"listed":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f66c6e7f15ef83fb","entry":"create_power_law_matrix","repo":"f-dangel/curvlinops","repo_kind":"listed","path":"docs/examples/basic_usage/example_trace_diagonal_estimation.py","file_url":"https://github.com/f-dangel/curvlinops/blob/HEAD/docs/examples/basic_usage/example_trace_diagonal_estimation.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f66c6e7f15ef83fb"}},{"code_sha256_prefix":"0f3de05d65a6ef31","entry":"relative_l_inf_error","repo":"f-dangel/curvlinops","repo_kind":"listed","path":"docs/examples/basic_usage/example_trace_diagonal_estimation.py","file_url":"https://github.com/f-dangel/curvlinops/blob/HEAD/docs/examples/basic_usage/example_trace_diagonal_estimation.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f3de05d65a6ef31"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}