{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dynamic-sparse-training-for-deep","title":"Dynamic Sparse Training for Deep Reinforcement Learning","arxiv_id":"2106.04217","date":"2021-06-08","proceeding":null,"authors":["Ghada Sokar","Elena Mocanu","Decebal Constantin Mocanu","Mykola Pechenizkiy","Peter Stone"],"abstract":"Deep reinforcement learning (DRL) agents are trained through trial-and-error interactions with the environment. This leads to a long training time for dense neural networks to achieve good performance. Hence, prohibitive computation and memory resources are consumed. Recently, learning efficient DRL agents has received increasing attention. Yet, current methods focus on accelerating inference time. In this paper, we introduce for the first time a dynamic sparse training approach for deep reinforcement learning to accelerate the training process. The proposed approach trains a sparse neural network from scratch and dynamically adapts its topology to the changing data distribution during training. Experiments on continuous control tasks show that our dynamic sparse agents achieve higher performance than the equivalent dense methods, reduce the parameter count and floating-point operations (FLOPs) by 50%, and have a faster learning speed that enables reaching the performance of dense agents with 40-50% reduction in the training steps.","url_abs":"https://arxiv.org/abs/2106.04217v3","url_pdf":"https://arxiv.org/pdf/2106.04217v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dynamic-sparse-training-for-deep","repo_url":"https://github.com/GhadaSokar/Dynamic-Sparse-Training-for-Deep-Reinforcement-Learning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"continuous-control","task_name":"Continuous Control"},{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"openai-gym","task_name":"OpenAI Gym"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"continuous-control","task_name":"continuous-control"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.04217","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.04217"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/GhadaSokar/Dynamic-Sparse-Training-for-Deep-Reinforcement-Learning","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":1,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"09540a28779b66ce","entry":"find_first_pos","repo":"GhadaSokar/Dynamic-Sparse-Training-for-Deep-Reinforcement-Learning","repo_kind":"official","path":"sparse_utils.py","file_url":"https://github.com/GhadaSokar/Dynamic-Sparse-Training-for-Deep-Reinforcement-Learning/blob/HEAD/sparse_utils.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"09540a28779b66ce"}},{"code_sha256_prefix":"20d5638125e09aa6","entry":"initializeEpsilonWeightsMask","repo":"GhadaSokar/Dynamic-Sparse-Training-for-Deep-Reinforcement-Learning","repo_kind":"official","path":"sparse_utils.py","file_url":"https://github.com/GhadaSokar/Dynamic-Sparse-Training-for-Deep-Reinforcement-Learning/blob/HEAD/sparse_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"20d5638125e09aa6"}},{"code_sha256_prefix":"e076d7fbad2f9ed8","entry":"initializeSparsityLevelWeightMask","repo":"GhadaSokar/Dynamic-Sparse-Training-for-Deep-Reinforcement-Learning","repo_kind":"official","path":"sparse_utils.py","file_url":"https://github.com/GhadaSokar/Dynamic-Sparse-Training-for-Deep-Reinforcement-Learning/blob/HEAD/sparse_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e076d7fbad2f9ed8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}