{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/challenges-in-high-dimensional-reinforcement","title":"Challenges in High-dimensional Reinforcement Learning with Evolution Strategies","arxiv_id":"1806.01224","date":"2018-06-04","proceeding":null,"authors":["Nils Müller","Tobias Glasmachers"],"abstract":"Evolution Strategies (ESs) have recently become popular for training deep\nneural networks, in particular on reinforcement learning tasks, a special form\nof controller design. Compared to classic problems in continuous direct search,\ndeep networks pose extremely high-dimensional optimization problems, with many\nthousands or even millions of variables. In addition, many control problems\ngive rise to a stochastic fitness function. Considering the relevance of the\napplication, we study the suitability of evolution strategies for\nhigh-dimensional, stochastic problems. Our results give insights into which\nalgorithmic mechanisms of modern ES are of value for the class of problems at\nhand, and they reveal principled limitations of the approach. They are in line\nwith our theoretical understanding of ESs. We show that combining ESs that\noffer reduced internal algorithm cost with uncertainty handling techniques\nyields promising methods for this class of problems.","url_abs":"http://arxiv.org/abs/1806.01224v2","url_pdf":"http://arxiv.org/pdf/1806.01224v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"challenges-in-high-dimensional-reinforcement","repo_url":"https://github.com/NiMlr/High-Dim-ES-RL","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"high","task_name":"Vocal Bursts Intensity Prediction"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1806.01224","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1806.01224"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/NiMlr/High-Dim-ES-RL","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f174478dbf156961","entry":"sigmoidal","repo":"NiMlr/High-Dim-ES-RL","repo_kind":"official","path":"benchmarkfunctions.py","file_url":"https://github.com/NiMlr/High-Dim-ES-RL/blob/HEAD/benchmarkfunctions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f174478dbf156961"}},{"code_sha256_prefix":"8eb7abd51c7bf2d3","entry":"sphere","repo":"NiMlr/High-Dim-ES-RL","repo_kind":"official","path":"benchmarkfunctions.py","file_url":"https://github.com/NiMlr/High-Dim-ES-RL/blob/HEAD/benchmarkfunctions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8eb7abd51c7bf2d3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}