{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/reducing-the-training-time-of-neural-networks","title":"Reducing the Training Time of Neural Networks by Partitioning","arxiv_id":"1511.02954","date":"2015-11-10","proceeding":null,"authors":["Conrado S. Miranda","Fernando J. Von Zuben"],"abstract":"This paper presents a new method for pre-training neural networks that can\ndecrease the total training time for a neural network while maintaining the\nfinal performance, which motivates its use on deep neural networks. By\npartitioning the training task in multiple training subtasks with sub-models,\nwhich can be performed independently and in parallel, it is shown that the size\nof the sub-models reduces almost quadratically with the number of subtasks\ncreated, quickly scaling down the sub-models used for the pre-training. The\nsub-models are then merged to provide a pre-trained initial set of weights for\nthe original model. The proposed method is independent of the other aspects of\nthe training, such as architecture of the neural network, training method, and\nobjective, making it compatible with a wide range of existing approaches. The\nspeedup without loss of performance is validated experimentally on MNIST and on\nCIFAR10 data sets, also showing that even performing the subtasks sequentially\ncan decrease the training time. Moreover, we show that larger models may\npresent higher speedups and conjecture about the benefits of the method in\ndistributed learning systems.","url_abs":"http://arxiv.org/abs/1511.02954v2","url_pdf":"http://arxiv.org/pdf/1511.02954v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"reducing-the-training-time-of-neural-networks","repo_url":"https://github.com/agongt408/vbranch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}