{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/how-dnns-break-the-curse-of-dimensionality","title":"How DNNs break the Curse of Dimensionality: Compositionality and Symmetry Learning","arxiv_id":"2407.05664","date":"2024-07-08","proceeding":null,"authors":["Arthur Jacot","Seok Hoan Choi","Yuxiao Wen"],"abstract":"We show that deep neural networks (DNNs) can efficiently learn any composition of functions with bounded $F_{1}$-norm, which allows DNNs to break the curse of dimensionality in ways that shallow networks cannot. More specifically, we derive a generalization bound that combines a covering number argument for compositionality, and the $F_{1}$-norm (or the related Barron norm) for large width adaptivity. We show that the global minimizer of the regularized loss of DNNs can fit for example the composition of two functions $f^{*}=h\\circ g$ from a small number of observations, assuming $g$ is smooth/regular and reduces the dimensionality (e.g. $g$ could be the modulo map of the symmetries of $f^{*}$), so that $h$ can be learned in spite of its low regularity. The measures of regularity we consider is the Sobolev norm with different levels of differentiability, which is well adapted to the $F_{1}$ norm. We compute scaling laws empirically and observe phase transitions depending on whether $g$ or $h$ is harder to learn, as predicted by our theory.","url_abs":"https://arxiv.org/abs/2407.05664v1","url_pdf":"https://arxiv.org/pdf/2407.05664v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"how-dnns-break-the-curse-of-dimensionality","repo_url":"https://github.com/shc443/coveringnumber_gb","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2407.05664","atlas_url":"https://app.syntology.ai/?focus=2407.05664","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.05664"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shc443/coveringnumber_gb","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"66d7b3dcb7bee596","entry":"bessel_k","repo":"shc443/coveringnumber_gb","repo_kind":"official","path":"MaternKernelTorch.py","file_url":"https://github.com/shc443/coveringnumber_gb/blob/HEAD/MaternKernelTorch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"66d7b3dcb7bee596"}},{"code_sha256_prefix":"b77a67b45a7d40c8","entry":"myExpFunc","repo":"shc443/coveringnumber_gb","repo_kind":"official","path":".ipynb_checkpoints/complexity-checkpoint.py","file_url":"https://github.com/shc443/coveringnumber_gb/blob/HEAD/.ipynb_checkpoints/complexity-checkpoint.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b77a67b45a7d40c8"}},{"code_sha256_prefix":"e850935d498e2118","entry":"nonlin","repo":"shc443/coveringnumber_gb","repo_kind":"official","path":".ipynb_checkpoints/complexity-checkpoint.py","file_url":"https://github.com/shc443/coveringnumber_gb/blob/HEAD/.ipynb_checkpoints/complexity-checkpoint.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e850935d498e2118"}},{"code_sha256_prefix":"5c61a10c18b5e2bc","entry":"load_compositional_dataset","repo":"shc443/coveringnumber_gb","repo_kind":"official","path":"cleaning/curse_dimensionality_sim/load_huggingface_data.py","file_url":"https://github.com/shc443/coveringnumber_gb/blob/HEAD/cleaning/curse_dimensionality_sim/load_huggingface_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5c61a10c18b5e2bc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}