{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/topology-of-deep-neural-networks","title":"Topology of deep neural networks","arxiv_id":"2004.06093","date":"2020-04-13","proceeding":null,"authors":["Gregory Naitzat","Andrey Zhitnikov","Lek-Heng Lim"],"abstract":"We study how the topology of a data set $M = M_a \\cup M_b \\subseteq \\mathbb{R}^d$, representing two classes $a$ and $b$ in a binary classification problem, changes as it passes through the layers of a well-trained neural network, i.e., with perfect accuracy on training set and near-zero generalization error ($\\approx 0.01\\%$). The goal is to shed light on two mysteries in deep neural networks: (i) a nonsmooth activation function like ReLU outperforms a smooth one like hyperbolic tangent; (ii) successful neural network architectures rely on having many layers, even though a shallow network can approximate any function arbitrary well. We performed extensive experiments on the persistent homology of a wide range of point cloud data sets, both real and simulated. The results consistently demonstrate the following: (1) Neural networks operate by changing topology, transforming a topologically complicated data set into a topologically simple one as it passes through the layers. No matter how complicated the topology of $M$ we begin with, when passed through a well-trained neural network $f : \\mathbb{R}^d \\to \\mathbb{R}^p$, there is a vast reduction in the Betti numbers of both components $M_a$ and $M_b$; in fact they nearly always reduce to their lowest possible values: $\\beta_k\\bigl(f(M_i)\\bigr) = 0$ for $k \\ge 1$ and $\\beta_0\\bigl(f(M_i)\\bigr) = 1$, $i =a, b$. Furthermore, (2) the reduction in Betti numbers is significantly faster for ReLU activation than hyperbolic tangent activation as the former defines nonhomeomorphic maps that change topology, whereas the latter defines homeomorphic maps that preserve topology. Lastly, (3) shallow and deep networks transform data sets differently -- a shallow network operates mainly through changing geometry and changes topology only in its final layers, a deep one spreads topological changes more evenly across all layers.","url_abs":"https://arxiv.org/abs/2004.06093v1","url_pdf":"https://arxiv.org/pdf/2004.06093v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"topology-of-deep-neural-networks","repo_url":"https://github.com/HLovisiEnnes/NetVisu","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"binary-classification","task_name":"Binary Classification"}],"methods":[{"method_slug":"relu","method_name":"ReLU"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2004.06093","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2004.06093"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/HLovisiEnnes/NetVisu","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":4},"by_repo_kind":{"listed":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"dfd7d9e8bf3967d0","entry":"circle_inside_torus","repo":"HLovisiEnnes/NetVisu","repo_kind":"listed","path":"datasets.py","file_url":"https://github.com/HLovisiEnnes/NetVisu/blob/HEAD/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dfd7d9e8bf3967d0"}},{"code_sha256_prefix":"5dfe8d1a61783de1","entry":"entangled_four_circles","repo":"HLovisiEnnes/NetVisu","repo_kind":"listed","path":"datasets.py","file_url":"https://github.com/HLovisiEnnes/NetVisu/blob/HEAD/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5dfe8d1a61783de1"}},{"code_sha256_prefix":"318474e19c34f4a4","entry":"entangled_two_circles","repo":"HLovisiEnnes/NetVisu","repo_kind":"listed","path":"datasets.py","file_url":"https://github.com/HLovisiEnnes/NetVisu/blob/HEAD/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"318474e19c34f4a4"}},{"code_sha256_prefix":"4c2ae2693221ea30","entry":"feedforward","repo":"HLovisiEnnes/NetVisu","repo_kind":"listed","path":"NetVisu.py","file_url":"https://github.com/HLovisiEnnes/NetVisu/blob/HEAD/NetVisu.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4c2ae2693221ea30"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}