{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-neuroevolution-genetic-algorithms-are-a","title":"Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning","arxiv_id":"1712.06567","date":"2017-12-18","proceeding":null,"authors":["Felipe Petroski Such","Vashisht Madhavan","Edoardo Conti","Joel Lehman","Kenneth O. Stanley","Jeff Clune"],"abstract":"Deep artificial neural networks (DNNs) are typically trained via\ngradient-based learning algorithms, namely backpropagation. Evolution\nstrategies (ES) can rival backprop-based algorithms such as Q-learning and\npolicy gradients on challenging deep reinforcement learning (RL) problems.\nHowever, ES can be considered a gradient-based algorithm because it performs\nstochastic gradient descent via an operation similar to a finite-difference\napproximation of the gradient. That raises the question of whether\nnon-gradient-based evolutionary algorithms can work at DNN scales. Here we\ndemonstrate they can: we evolve the weights of a DNN with a simple,\ngradient-free, population-based genetic algorithm (GA) and it performs well on\nhard deep RL problems, including Atari and humanoid locomotion. The Deep GA\nsuccessfully evolves networks with over four million free parameters, the\nlargest neural networks ever evolved with a traditional evolutionary algorithm.\nThese results (1) expand our sense of the scale at which GAs can operate, (2)\nsuggest intriguingly that in some cases following the gradient is not the best\nchoice for optimizing performance, and (3) make immediately available the\nmultitude of neuroevolution techniques that improve performance. We demonstrate\nthe latter by showing that combining DNNs with novelty search, which encourages\nexploration on tasks with deceptive or sparse reward functions, can solve a\nhigh-dimensional problem on which reward-maximizing algorithms (e.g.\\ DQN, A3C,\nES, and the GA) fail. Additionally, the Deep GA is faster than ES, A3C, and DQN\n(it can train Atari in ${\\raise.17ex\\hbox{$\\scriptstyle\\sim$}}$4 hours on one\ndesktop or ${\\raise.17ex\\hbox{$\\scriptstyle\\sim$}}$1 hour distributed on 720\ncores), and enables a state-of-the-art, up to 10,000-fold compact encoding\ntechnique.","url_abs":"http://arxiv.org/abs/1712.06567v3","url_pdf":"http://arxiv.org/pdf/1712.06567v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-neuroevolution-genetic-algorithms-are-a","repo_url":"https://github.com/Bibyutatsu/Self_Driving_Car","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"deep-neuroevolution-genetic-algorithms-are-a","repo_url":"https://github.com/Frager/Deep-Neuroevolution-In-SC2LE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"deep-neuroevolution-genetic-algorithms-are-a","repo_url":"https://github.com/ZainRaza14/Neuro-Evolution","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"deep-neuroevolution-genetic-algorithms-are-a","repo_url":"https://github.com/brando130/CSharp_NN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"deep-neuroevolution-genetic-algorithms-are-a","repo_url":"https://github.com/kevin5naug/summer_project","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"deep-neuroevolution-genetic-algorithms-are-a","repo_url":"https://github.com/opent03/neural_cars","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"deep-neuroevolution-genetic-algorithms-are-a","repo_url":"https://github.com/optimization-toolbox/DNE4py","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"deep-neuroevolution-genetic-algorithms-are-a","repo_url":"https://github.com/optimization-toolbox/deep-neuroevolution-mpi4py","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"deep-neuroevolution-genetic-algorithms-are-a","repo_url":"https://github.com/potatoehacker/ArticleDeepNeuroEvolution","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"deep-neuroevolution-genetic-algorithms-are-a","repo_url":"https://github.com/theneuralbeing/neuroevolution","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"deep-neuroevolution-genetic-algorithms-are-a","repo_url":"https://github.com/uber-common/deep-neuroevolution","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"deep-neuroevolution-genetic-algorithms-are-a","repo_url":"https://github.com/uber-research/deep-neuroevolution","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"evolutionary-algorithms","task_name":"Evolutionary Algorithms"},{"task_slug":"q-learning","task_name":"Q-Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[{"method_slug":"a3c","method_name":"A3C"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dqn","method_name":"DQN"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"entropy-regularization","method_name":"Entropy Regularization"},{"method_slug":"q-learning","method_name":"Q-Learning"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1712.06567","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1712.06567"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Bibyutatsu/Self_Driving_Car","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ZainRaza14/Neuro-Evolution","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Frager/Deep-Neuroevolution-In-SC2LE","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/theneuralbeing/neuroevolution","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kevin5naug/summer_project","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/uber-common/deep-neuroevolution","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/brando130/CSharp_NN","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/potatoehacker/ArticleDeepNeuroEvolution","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/optimization-toolbox/deep-neuroevolution-mpi4py","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/opent03/neural_cars","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/uber-research/deep-neuroevolution","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/optimization-toolbox/DNE4py","reach":null}],"summary":{"ran_violates":1,"ran_fixture":1,"ran_honours":1,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"listed":{"samples":2,"ran":2,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"c8c2960f6687b93c","entry":"compute_centered_ranks","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"c8c2960f6687b93c"}},{"code_sha256_prefix":"e2d29b359d6b9487","entry":"compute_ranks","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"e2d29b359d6b9487"}},{"code_sha256_prefix":"1fc06388f906350c","entry":"objective","repo":"optimization-toolbox/DNE4py","repo_kind":"listed","path":"DNE4py/version_v2/new_tutorials/tutorials2/pp_run.py","file_url":"https://github.com/optimization-toolbox/DNE4py/blob/HEAD/DNE4py/version_v2/new_tutorials/tutorials2/pp_run.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"1fc06388f906350c"}},{"code_sha256_prefix":"dac1c1782a9dfe5d","entry":"uncompress_model","repo":"Bibyutatsu/Self_Driving_Car","repo_kind":"listed","path":"ai.py","file_url":"https://github.com/Bibyutatsu/Self_Driving_Car/blob/HEAD/ai.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dac1c1782a9dfe5d"}},{"code_sha256_prefix":"703807555aa85c6d","entry":"make_session","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"703807555aa85c6d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}