{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bayesian-optimization-of-hyper-parameters-in","title":"Bayesian optimization of hyper-parameters in reservoir computing","arxiv_id":"1611.05193","date":"2016-11-16","proceeding":null,"authors":["Jan Yperman","Thijs Becker"],"abstract":"We describe a method for searching the optimal hyper-parameters in reservoir\ncomputing, which consists of a Gaussian process with Bayesian optimization. It\nprovides an alternative to other frequently used optimization methods such as\ngrid, random, or manual search. In addition to a set of optimal\nhyper-parameters, the method also provides a probability distribution of the\ncost function as a function of the hyper-parameters. We apply this method to\ntwo types of reservoirs: nonlinear delay nodes and echo state networks. It\nshows excellent performance on all considered benchmarks, either matching or\nsignificantly surpassing results found in the literature. In general, the\nalgorithm achieves optimal results in fewer iterations when compared to other\noptimization methods. We have optimized up to six hyper-parameters\nsimultaneously, which would have been infeasible using, e.g., grid search. Due\nto its automated nature, this method significantly reduces the need for expert\nknowledge when optimizing the hyper-parameters in reservoir computing. Existing\nsoftware libraries for Bayesian optimization, such as Spearmint, make the\nimplementation of the algorithm straightforward. A fork of the Spearmint\nframework along with a tutorial on how to use it in practice is available at\nhttps://bitbucket.org/uhasseltmachinelearning/spearmint/","url_abs":"http://arxiv.org/abs/1611.05193v3","url_pdf":"http://arxiv.org/pdf/1611.05193v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bayesian-optimization-of-hyper-parameters-in","repo_url":"https://bitbucket.org/uhasseltmachinelearning/spearmint","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"bayesian-optimization-of-hyper-parameters-in","repo_url":"https://github.com/rednotion/parallel_esn_web","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"bayesian-optimization","task_name":"Bayesian Optimization"}],"methods":[{"method_slug":"gaussian-process","method_name":"Gaussian Process"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}