{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/neural-collapse-with-normalized-features-a","title":"Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold","arxiv_id":"2209.09211","date":"2022-09-19","proceeding":null,"authors":["Can Yaras","Peng Wang","Zhihui Zhu","Laura Balzano","Qing Qu"],"abstract":"When training overparameterized deep networks for classification tasks, it has been widely observed that the learned features exhibit a so-called \"neural collapse\" phenomenon. More specifically, for the output features of the penultimate layer, for each class the within-class features converge to their means, and the means of different classes exhibit a certain tight frame structure, which is also aligned with the last layer's classifier. As feature normalization in the last layer becomes a common practice in modern representation learning, in this work we theoretically justify the neural collapse phenomenon for normalized features. Based on an unconstrained feature model, we simplify the empirical loss function in a multi-class classification task into a nonconvex optimization problem over the Riemannian manifold by constraining all features and classifiers over the sphere. In this context, we analyze the nonconvex landscape of the Riemannian optimization problem over the product of spheres, showing a benign global landscape in the sense that the only global minimizers are the neural collapse solutions while all other critical points are strict saddles with negative curvature. Experimental results on practical deep networks corroborate our theory and demonstrate that better representations can be learned faster via feature normalization.","url_abs":"https://arxiv.org/abs/2209.09211v2","url_pdf":"https://arxiv.org/pdf/2209.09211v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"neural-collapse-with-normalized-features-a","repo_url":"https://github.com/cjyaras/normalized-neural-collapse","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"multi-class-classification","task_name":"Multi-class Classification"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"riemannian-optimization","task_name":"Riemannian optimization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2209.09211","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2209.09211"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/cjyaras/normalized-neural-collapse","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1caa9eb23050c88a","entry":"ResNet18VariableWidth","repo":"cjyaras/normalized-neural-collapse","repo_kind":"official","path":"models/resnet_vw.py","file_url":"https://github.com/cjyaras/normalized-neural-collapse/blob/HEAD/models/resnet_vw.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1caa9eb23050c88a"}},{"code_sha256_prefix":"f88cd35cd453165b","entry":"create_dataset","repo":"cjyaras/normalized-neural-collapse","repo_kind":"official","path":"datasets.py","file_url":"https://github.com/cjyaras/normalized-neural-collapse/blob/HEAD/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f88cd35cd453165b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}