{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/progressive-learning-for-systematic-design-of","title":"Progressive Learning for Systematic Design of Large Neural Networks","arxiv_id":"1710.08177","date":"2017-10-23","proceeding":null,"authors":["Saikat Chatterjee","Alireza M. Javid","Mostafa Sadeghi","Partha P. Mitra","Mikael Skoglund"],"abstract":"We develop an algorithm for systematic design of a large artificial neural\nnetwork using a progression property. We find that some non-linear functions,\nsuch as the rectifier linear unit and its derivatives, hold the property. The\nsystematic design addresses the choice of network size and regularization of\nparameters. The number of nodes and layers in network increases in progression\nwith the objective of consistently reducing an appropriate cost. Each layer is\noptimized at a time, where appropriate parameters are learned using convex\noptimization. Regularization parameters for convex optimization do not need a\nsignificant manual effort for tuning. We also use random instances for some\nweight matrices, and that helps to reduce the number of parameters we learn.\nThe developed network is expected to show good generalization power due to\nappropriate regularization and use of random weights in the layers. This\nexpectation is verified by extensive experiments for classification and\nregression problems, using standard databases.","url_abs":"http://arxiv.org/abs/1710.08177v1","url_pdf":"http://arxiv.org/pdf/1710.08177v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"progressive-learning-for-systematic-design-of","repo_url":"https://github.com/viebboy/HeMLGOP","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}