{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/reinforcement-learning-with-augmented-data","title":"Reinforcement Learning with Augmented Data","arxiv_id":"2004.14990","date":"2020-04-30","proceeding":"NeurIPS 2020 12","authors":["Michael Laskin","Kimin Lee","Adam Stooke","Lerrel Pinto","Pieter Abbeel","Aravind Srinivas"],"abstract":"Learning from visual observations is a fundamental yet challenging problem in Reinforcement Learning (RL). Although algorithmic advances combined with convolutional neural networks have proved to be a recipe for success, current methods are still lacking on two fronts: (a) data-efficiency of learning and (b) generalization to new environments. To this end, we present Reinforcement Learning with Augmented Data (RAD), a simple plug-and-play module that can enhance most RL algorithms. We perform the first extensive study of general data augmentations for RL on both pixel-based and state-based inputs, and introduce two new data augmentations - random translate and random amplitude scale. We show that augmentations such as random translate, crop, color jitter, patch cutout, random convolutions, and amplitude scale can enable simple RL algorithms to outperform complex state-of-the-art methods across common benchmarks. RAD sets a new state-of-the-art in terms of data-efficiency and final performance on the DeepMind Control Suite benchmark for pixel-based control as well as OpenAI Gym benchmark for state-based control. We further demonstrate that RAD significantly improves test-time generalization over existing methods on several OpenAI ProcGen benchmarks. Our RAD module and training code are available at https://www.github.com/MishaLaskin/rad.","url_abs":"https://arxiv.org/abs/2004.14990v5","url_pdf":"https://arxiv.org/pdf/2004.14990v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"reinforcement-learning-with-augmented-data","repo_url":"https://github.com/MishaLaskin/rad","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"reinforcement-learning-with-augmented-data","repo_url":"https://github.com/KarlXing/RL-Visual-Continuous-Control","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"openai-gym","task_name":"OpenAI Gym"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"entropy-regularization","method_name":"Entropy Regularization"},{"method_slug":"experience-replay","method_name":"Experience Replay"},{"method_slug":"ppo","method_name":"PPO"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"soft-actor-critic","method_name":"Soft Actor Critic"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2004.14990","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2004.14990"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/MishaLaskin/rad","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/vitchyr/rlkit","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/KarlXing/RL-Visual-Continuous-Control","reach":null}],"summary":{"ran":8,"ran_fixture":9,"ran_draft_wrong":3,"ran_honours":1,"unverified":3},"by_repo_kind":{"named_in_paper":{"samples":21,"ran":18,"repositories":1},"listed":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":21,"samples":[{"code_sha256_prefix":"5aaa7c97de18af3a","entry":"Actor","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5aaa7c97de18af3a"}},{"code_sha256_prefix":"a551a31aced2e8ed","entry":"CURL","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a551a31aced2e8ed"}},{"code_sha256_prefix":"7eeb6c2c86ebd37d","entry":"ColorJitterLayer","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7eeb6c2c86ebd37d"}},{"code_sha256_prefix":"faa8d44dfeaada9d","entry":"Critic","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"faa8d44dfeaada9d"}},{"code_sha256_prefix":"8b06f5d9a377410d","entry":"IdentityEncoder","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8b06f5d9a377410d"}},{"code_sha256_prefix":"75ea19a7c991ff1d","entry":"PixelEncoder","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"75ea19a7c991ff1d"}},{"code_sha256_prefix":"4ef6619d0e3853e4","entry":"RAD","repo":"KarlXing/RL-Visual-Continuous-Control","repo_kind":"listed","path":"src/agent/rad.py","file_url":"https://github.com/KarlXing/RL-Visual-Continuous-Control/blob/HEAD/src/agent/rad.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4ef6619d0e3853e4"}},{"code_sha256_prefix":"bf3db7327a139e28","entry":"SAC","repo":"KarlXing/RL-Visual-Continuous-Control","repo_kind":"listed","path":"src/agent/rad.py","file_url":"https://github.com/KarlXing/RL-Visual-Continuous-Control/blob/HEAD/src/agent/rad.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bf3db7327a139e28"}},{"code_sha256_prefix":"31e6960f8b942b0d","entry":"center_crop_image","repo":"KarlXing/RL-Visual-Continuous-Control","repo_kind":"listed","path":"src/agent/rad.py","file_url":"https://github.com/KarlXing/RL-Visual-Continuous-Control/blob/HEAD/src/agent/rad.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"31e6960f8b942b0d"}},{"code_sha256_prefix":"1bb28d88e7ae1e71","entry":"grayscale","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1bb28d88e7ae1e71"}},{"code_sha256_prefix":"718e1902f65f1be2","entry":"hsv2rgb","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"718e1902f65f1be2"}},{"code_sha256_prefix":"3ff1a7cb98f67025","entry":"make_encoder","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3ff1a7cb98f67025"}},{"code_sha256_prefix":"b1cb15a87e0863be","entry":"no_aug","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b1cb15a87e0863be"}},{"code_sha256_prefix":"3b7275db7d30b2b9","entry":"random_convolution","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3b7275db7d30b2b9"}},{"code_sha256_prefix":"cc205ddaca3bfbbc","entry":"random_crop","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cc205ddaca3bfbbc"}},{"code_sha256_prefix":"417e34ed0818a71b","entry":"random_cutout","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"417e34ed0818a71b"}},{"code_sha256_prefix":"d4841127aa83b755","entry":"random_cutout_color","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d4841127aa83b755"}},{"code_sha256_prefix":"0c0e4ec30cabf972","entry":"random_flip","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0c0e4ec30cabf972"}},{"code_sha256_prefix":"ef56bb771298e9ef","entry":"random_grayscale","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ef56bb771298e9ef"}},{"code_sha256_prefix":"77241bfa75e23abd","entry":"random_translate","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"77241bfa75e23abd"}},{"code_sha256_prefix":"1c24bb2fb72c4f9d","entry":"rgb2hsv","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1c24bb2fb72c4f9d"}},{"code_sha256_prefix":"62d57e42506581cb","entry":"RadSacAgent","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"62d57e42506581cb"}},{"code_sha256_prefix":"5a5ab3e704f9a4cd","entry":"random_color_jitter","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5a5ab3e704f9a4cd"}},{"code_sha256_prefix":"c0364ede43f86490","entry":"random_rotation","repo":"MishaLaskin/rad","repo_kind":"named_in_paper","path":"curl_sac.py","file_url":"https://github.com/MishaLaskin/rad/blob/HEAD/curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c0364ede43f86490"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}