{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/separation-and-concentration-in-deep-networks","title":"Separation and Concentration in Deep Networks","arxiv_id":"2012.10424","date":"2020-12-18","proceeding":null,"authors":["John Zarka","Florentin Guth","Stéphane Mallat"],"abstract":"Numerical experiments demonstrate that deep neural network classifiers progressively separate class distributions around their mean, achieving linear separability on the training set, and increasing the Fisher discriminant ratio. We explain this mechanism with two types of operators. We prove that a rectifier without biases applied to sign-invariant tight frames can separate class means and increase Fisher ratios. On the opposite, a soft-thresholding on tight frames can reduce within-class variabilities while preserving class means. Variance reduction bounds are proved for Gaussian mixture models. For image classification, we show that separation of class means can be achieved with rectified wavelet tight frames that are not learned. It defines a scattering transform. Learning $1 \\times 1$ convolutional tight frames along scattering channels and applying a soft-thresholding reduces within-class variabilities. The resulting scattering network reaches the classification accuracy of ResNet-18 on CIFAR-10 and ImageNet, with fewer layers and no learned biases.","url_abs":"https://arxiv.org/abs/2012.10424v2","url_pdf":"https://arxiv.org/pdf/2012.10424v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"separation-and-concentration-in-deep-networks","repo_url":"https://github.com/iclr2021-paper1937/separation_concentration_deepnets","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"separation-and-concentration-in-deep-networks","repo_url":"https://github.com/j-zarka/separation_concentration_deepnets","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2012.10424","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2012.10424"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/j-zarka/separation_concentration_deepnets","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/iclr2021-paper1937/separation_concentration_deepnets","reach":{"status":"unanswered"}}],"summary":{"ran_draft_wrong":4},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"eac5047cfece5c96","entry":"absolute","repo":"j-zarka/separation_concentration_deepnets","repo_kind":"official","path":"models/Analysis.py","file_url":"https://github.com/j-zarka/separation_concentration_deepnets/blob/HEAD/models/Analysis.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"eac5047cfece5c96"}},{"code_sha256_prefix":"e2185e2723a6f7ea","entry":"dispatch_non_linearity","repo":"j-zarka/separation_concentration_deepnets","repo_kind":"official","path":"models/Analysis.py","file_url":"https://github.com/j-zarka/separation_concentration_deepnets/blob/HEAD/models/Analysis.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"e2185e2723a6f7ea"}},{"code_sha256_prefix":"9a69065038cc553e","entry":"hook_fn","repo":"j-zarka/separation_concentration_deepnets","repo_kind":"official","path":"models/ScatNetAnalysis.py","file_url":"https://github.com/j-zarka/separation_concentration_deepnets/blob/HEAD/models/ScatNetAnalysis.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"9a69065038cc553e"}},{"code_sha256_prefix":"d5d46e42d48015f4","entry":"relu","repo":"j-zarka/separation_concentration_deepnets","repo_kind":"official","path":"models/Analysis.py","file_url":"https://github.com/j-zarka/separation_concentration_deepnets/blob/HEAD/models/Analysis.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"d5d46e42d48015f4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}