{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/parallel-training-considered-harmful","title":"\"Parallel Training Considered Harmful?\": Comparing series-parallel and parallel feedforward network training","arxiv_id":"1706.07119","date":"2017-06-21","proceeding":null,"authors":["Antônio H. Ribeiro","Luis A. Aguirre"],"abstract":"Neural network models for dynamic systems can be trained either in parallel\nor in series-parallel configurations. Influenced by early arguments, several\npapers justify the choice of series-parallel rather than parallel configuration\nclaiming it has a lower computational cost, better stability properties during\ntraining and provides more accurate results. Other published results, on the\nother hand, defend parallel training as being more robust and capable of\nyielding more accu- rate long-term predictions. The main contribution of this\npaper is to present a study comparing both methods under the same unified\nframework. We focus on three aspects: i) robustness of the estimation in the\npresence of noise; ii) computational cost; and, iii) convergence. A unifying\nmathematical framework and simulation studies show situations where each\ntraining method provides better validation results, being parallel training\nbetter in what is believed to be more realistic scenarios. An example using\nmeasured data seems to reinforce such claim. We also show, with a novel\ncomplexity analysis and numerical examples, that both methods have similar\ncomputational cost, being series series-parallel training, however, more\namenable to parallelization. Some informal discussion about stability and\nconvergence properties is presented and explored in the examples.","url_abs":"http://arxiv.org/abs/1706.07119v3","url_pdf":"http://arxiv.org/pdf/1706.07119v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"parallel-training-considered-harmful","repo_url":"https://github.com/antonior92/ParallelTrainingNN.jl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}