{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/automatic-noise-filtering-with-dynamic-sparse","title":"Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learning","arxiv_id":"2302.06548","date":"2023-02-13","proceeding":null,"authors":["Bram Grooten","Ghada Sokar","Shibhansh Dohare","Elena Mocanu","Matthew E. Taylor","Mykola Pechenizkiy","Decebal Constantin Mocanu"],"abstract":"Tomorrow's robots will need to distinguish useful information from noise when performing different tasks. A household robot for instance may continuously receive a plethora of information about the home, but needs to focus on just a small subset to successfully execute its current chore. Filtering distracting inputs that contain irrelevant data has received little attention in the reinforcement learning literature. To start resolving this, we formulate a problem setting in reinforcement learning called the $\\textit{extremely noisy environment}$ (ENE), where up to $99\\%$ of the input features are pure noise. Agents need to detect which features provide task-relevant information about the state of the environment. Consequently, we propose a new method termed $\\textit{Automatic Noise Filtering}$ (ANF), which uses the principles of dynamic sparse training in synergy with various deep reinforcement learning algorithms. The sparse input layer learns to focus its connectivity on task-relevant features, such that ANF-SAC and ANF-TD3 outperform standard SAC and TD3 by a large margin, while using up to $95\\%$ fewer weights. Furthermore, we devise a transfer learning setting for ENEs, by permuting all features of the environment after 1M timesteps to simulate the fact that other information sources can become relevant as the world evolves. Again, ANF surpasses the baselines in final performance and sample complexity. Our code is available at https://github.com/bramgrooten/automatic-noise-filtering","url_abs":"https://arxiv.org/abs/2302.06548v1","url_pdf":"https://arxiv.org/pdf/2302.06548v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"automatic-noise-filtering-with-dynamic-sparse","repo_url":"https://github.com/bramgrooten/automatic-noise-filtering","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"clipped-double-q-learning","method_name":"Clipped Double Q-learning"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dilated-convolution","method_name":"Dilated Convolution"},{"method_slug":"experience-replay","method_name":"Experience Replay"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"sac","method_name":"SAC"},{"method_slug":"td3","method_name":"TD3"},{"method_slug":"target-policy-smoothing","method_name":"Target Policy Smoothing"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2302.06548","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2302.06548"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bramgrooten/automatic-noise-filtering","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_violates":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ef2325b4dcee1c65","entry":"unravel_index","repo":"bramgrooten/automatic-noise-filtering","repo_kind":"official","path":"utils/target_network.py","file_url":"https://github.com/bramgrooten/automatic-noise-filtering/blob/HEAD/utils/target_network.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ef2325b4dcee1c65"}},{"code_sha256_prefix":"7cf031dd1dba224b","entry":"format_env","repo":"bramgrooten/automatic-noise-filtering","repo_kind":"official","path":"view_mujoco.py","file_url":"https://github.com/bramgrooten/automatic-noise-filtering/blob/HEAD/view_mujoco.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7cf031dd1dba224b"}},{"code_sha256_prefix":"cda5c99bd22814fd","entry":"format_optimizer_arg","repo":"bramgrooten/automatic-noise-filtering","repo_kind":"official","path":"view_mujoco.py","file_url":"https://github.com/bramgrooten/automatic-noise-filtering/blob/HEAD/view_mujoco.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cda5c99bd22814fd"}},{"code_sha256_prefix":"45abacb2907451e4","entry":"format_sparsity","repo":"bramgrooten/automatic-noise-filtering","repo_kind":"official","path":"view_mujoco.py","file_url":"https://github.com/bramgrooten/automatic-noise-filtering/blob/HEAD/view_mujoco.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"45abacb2907451e4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}