{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-semi-random-features-for-nonlinear","title":"Deep Semi-Random Features for Nonlinear Function Approximation","arxiv_id":"1702.08882","date":"2017-02-28","proceeding":null,"authors":["Kenji Kawaguchi","Bo Xie","Vikas Verma","Le Song"],"abstract":"We propose semi-random features for nonlinear function approximation. The\nflexibility of semi-random feature lies between the fully adjustable units in\ndeep learning and the random features used in kernel methods. For one hidden\nlayer models with semi-random features, we prove with no unrealistic\nassumptions that the model classes contain an arbitrarily good function as the\nwidth increases (universality), and despite non-convexity, we can find such a\ngood function (optimization theory) that generalizes to unseen new data\n(generalization bound). For deep models, with no unrealistic assumptions, we\nprove universal approximation ability, a lower bound on approximation error, a\npartial optimization guarantee, and a generalization bound. Depending on the\nproblems, the generalization bound of deep semi-random features can be\nexponentially better than the known bounds of deep ReLU nets; our\ngeneralization error bound can be independent of the depth, the number of\ntrainable weights as well as the input dimensionality. In experiments, we show\nthat semi-random features can match the performance of neural networks by using\nslightly more units, and it outperforms random features by using significantly\nfewer units. Moreover, we introduce a new implicit ensemble method by using\nsemi-random features.","url_abs":"http://arxiv.org/abs/1702.08882v7","url_pdf":"http://arxiv.org/pdf/1702.08882v7.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-semi-random-features-for-nonlinear","repo_url":"https://github.com/zixu1986/semi-random","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[],"methods":[{"method_slug":"relu","method_name":"ReLU"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}