{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scalable-training-of-artificial-neural","title":"Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science","arxiv_id":"1707.04780","date":"2017-07-15","proceeding":null,"authors":["Decebal Constantin Mocanu","Elena Mocanu","Peter Stone","Phuong H. Nguyen","Madeleine Gibescu","Antonio Liotta"],"abstract":"Through the success of deep learning in various domains, artificial neural\nnetworks are currently among the most used artificial intelligence methods.\nTaking inspiration from the network properties of biological neural networks\n(e.g. sparsity, scale-freeness), we argue that (contrary to general practice)\nartificial neural networks, too, should not have fully-connected layers. Here\nwe propose sparse evolutionary training of artificial neural networks, an\nalgorithm which evolves an initial sparse topology (Erd\\H{o}s-R\\'enyi random\ngraph) of two consecutive layers of neurons into a scale-free topology, during\nlearning. Our method replaces artificial neural networks fully-connected layers\nwith sparse ones before training, reducing quadratically the number of\nparameters, with no decrease in accuracy. We demonstrate our claims on\nrestricted Boltzmann machines, multi-layer perceptrons, and convolutional\nneural networks for unsupervised and supervised learning on 15 datasets. Our\napproach has the potential to enable artificial neural networks to scale up\nbeyond what is currently possible.","url_abs":"http://arxiv.org/abs/1707.04780v2","url_pdf":"http://arxiv.org/pdf/1707.04780v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scalable-training-of-artificial-neural","repo_url":"https://github.com/dcmocanu/sparse-evolutionary-artificial-neural-networks","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"scalable-training-of-artificial-neural","repo_url":"https://github.com/gru2/DoubleBlockSparse","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"sparse-learning","task_name":"Sparse Learning"}],"methods":[{"method_slug":"dst","method_name":"DST"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[{"slug":"dst","name":"DST","full_name":"Dynamic Sparse Training"},{"slug":"set","name":"SET","full_name":"Sparse Evolutionary Training"}],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1707.04780","atlas_url":"https://app.syntology.ai/?focus=1707.04780","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1707.04780"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dcmocanu/sparse-evolutionary-artificial-neural-networks","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gru2/DoubleBlockSparse","reach":{"status":"unanswered"}}],"summary":{"ran_draft_wrong":1,"ran_honours":2},"by_repo_kind":{},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"d1dd6b1240c82834","entry":"createSparseWeights","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"d1dd6b1240c82834"}},{"code_sha256_prefix":"09540a28779b66ce","entry":"find_first_pos","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"09540a28779b66ce"}},{"code_sha256_prefix":"6c42c3044030b535","entry":"find_last_pos","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"6c42c3044030b535"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}