{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/parameter-space-noise-for-exploration","title":"Parameter Space Noise for Exploration","arxiv_id":"1706.01905","date":"2017-06-06","proceeding":"ICLR 2018 1","authors":["Matthias Plappert","Rein Houthooft","Prafulla Dhariwal","Szymon Sidor","Richard Y. Chen","Xi Chen","Tamim Asfour","Pieter Abbeel","Marcin Andrychowicz"],"abstract":"Deep reinforcement learning (RL) methods generally engage in exploratory\nbehavior through noise injection in the action space. An alternative is to add\nnoise directly to the agent's parameters, which can lead to more consistent\nexploration and a richer set of behaviors. Methods such as evolutionary\nstrategies use parameter perturbations, but discard all temporal structure in\nthe process and require significantly more samples. Combining parameter noise\nwith traditional RL methods allows to combine the best of both worlds. We\ndemonstrate that both off- and on-policy methods benefit from this approach\nthrough experimental comparison of DQN, DDPG, and TRPO on high-dimensional\ndiscrete action environments as well as continuous control tasks. Our results\nshow that RL with parameter noise learns more efficiently than traditional RL\nwith action space noise and evolutionary strategies individually.","url_abs":"http://arxiv.org/abs/1706.01905v2","url_pdf":"http://arxiv.org/pdf/1706.01905v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"parameter-space-noise-for-exploration","repo_url":"https://github.com/JonasRSV/DDPG","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"parameter-space-noise-for-exploration","repo_url":"https://github.com/JonasRSV/DQNTensorflow","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"parameter-space-noise-for-exploration","repo_url":"https://github.com/JonasRSV/PGTensorflow","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"parameter-space-noise-for-exploration","repo_url":"https://github.com/aspk/Tensorflow_Deep_RL","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"parameter-space-noise-for-exploration","repo_url":"https://github.com/flowersteam/geppg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"parameter-space-noise-for-exploration","repo_url":"https://github.com/h-aboutalebi/SparceReward","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"parameter-space-noise-for-exploration","repo_url":"https://github.com/susan-amin/SparseBaseline1","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"parameter-space-noise-for-exploration","repo_url":"https://github.com/tensorflow/models","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"parameter-space-noise-for-exploration","repo_url":"https://github.com/tensorflow/models/tree/master/research/deep_contextual_bandits","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"parameter-space-noise-for-exploration","repo_url":"https://github.com/hill-a/stable-baselines","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"continuous-control","task_name":"Continuous Control"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"continuous-control","task_name":"continuous-control"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"ddpg","method_name":"DDPG"},{"method_slug":"dqn","method_name":"DQN"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"experience-replay","method_name":"Experience Replay"},{"method_slug":"q-learning","method_name":"Q-Learning"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"trpo","method_name":"TRPO"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1706.01905","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1706.01905"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hill-a/stable-baselines","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tensorflow/models","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/JonasRSV/PGTensorflow","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/flowersteam/geppg","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/aspk/Tensorflow_Deep_RL","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tensorflow/models/tree/master/research/deep_contextual_bandits","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/JonasRSV/DQNTensorflow","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/h-aboutalebi/SparceReward","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/JonasRSV/DDPG","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/susan-amin/SparseBaseline1","reach":null}],"summary":{"ran_violates":1,"ran_fixture":1,"ran_honours":1,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"listed":{"samples":5,"ran":4,"repositories":3}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"7e13cf9f394dd7cc","entry":"action_modifier","repo":"JonasRSV/DDPG","repo_kind":"listed","path":"pendelum_demo.py","file_url":"https://github.com/JonasRSV/DDPG/blob/HEAD/pendelum_demo.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7e13cf9f394dd7cc"}},{"code_sha256_prefix":"c80e4e6524f7b68e","entry":"discount","repo":"aspk/Tensorflow_Deep_RL","repo_kind":"listed","path":"param_noise_mlp.py","file_url":"https://github.com/aspk/Tensorflow_Deep_RL/blob/HEAD/param_noise_mlp.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c80e4e6524f7b68e"}},{"code_sha256_prefix":"546e7278ab5fc8cd","entry":"extract_performances","repo":"flowersteam/geppg","repo_kind":"listed","path":"scripts/extract_results.py","file_url":"https://github.com/flowersteam/geppg/blob/HEAD/scripts/extract_results.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"546e7278ab5fc8cd"}},{"code_sha256_prefix":"032931d3060ffcf3","entry":"play_policy","repo":"flowersteam/geppg","repo_kind":"listed","path":"gep.py","file_url":"https://github.com/flowersteam/geppg/blob/HEAD/gep.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"032931d3060ffcf3"}},{"code_sha256_prefix":"97cda2f13eccfcc4","entry":"offline_evaluations","repo":"flowersteam/geppg","repo_kind":"listed","path":"gep.py","file_url":"https://github.com/flowersteam/geppg/blob/HEAD/gep.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"97cda2f13eccfcc4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}