{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-optimization-of-echo-state-networks","title":"Efficient Optimization of Echo State Networks for Time Series Datasets","arxiv_id":"1903.05071","date":"2019-03-12","proceeding":null,"authors":["Jacob Reinier Maat","Nikos Gianniotis","Pavlos Protopapas"],"abstract":"Echo State Networks (ESNs) are recurrent neural networks that only train\ntheir output layer, thereby precluding the need to backpropagate gradients\nthrough time, which leads to significant computational gains. Nevertheless, a\ncommon issue in ESNs is determining its hyperparameters, which are crucial in\ninstantiating a well performing reservoir, but are often set manually or using\nheuristics. In this work we optimize the ESN hyperparameters using Bayesian\noptimization which, given a limited budget of function evaluations, outperforms\na grid search strategy. In the context of large volumes of time series data,\nsuch as light curves in the field of astronomy, we can further reduce the\noptimization cost of ESNs. In particular, we wish to avoid tuning\nhyperparameters per individual time series as this is costly; instead, we want\nto find ESNs with hyperparameters that perform well not just on individual time\nseries but rather on groups of similar time series without sacrificing\npredictive performance significantly. This naturally leads to a notion of\nclusters, where each cluster is represented by an ESN tuned to model a group of\ntime series of similar temporal behavior. We demonstrate this approach both on\nsynthetic datasets and real world light curves from the MACHO survey. We show\nthat our approach results in a significant reduction in the number of ESN\nmodels required to model a whole dataset, while retaining predictive\nperformance for the series in each cluster.","url_abs":"http://arxiv.org/abs/1903.05071v1","url_pdf":"http://arxiv.org/pdf/1903.05071v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-optimization-of-echo-state-networks","repo_url":"https://github.com/1Reinier/Reservoir","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"efficient-optimization-of-echo-state-networks","repo_url":"https://github.com/blindedjoy/RcTorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"astronomy","task_name":"Astronomy"},{"task_slug":"bayesian-optimization","task_name":"Bayesian Optimization"},{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series","task_name":"Time Series Analysis"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}