{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/emergence-of-in-context-reinforcement","title":"Emergence of In-Context Reinforcement Learning from Noise Distillation","arxiv_id":"2312.12275","date":"2023-12-19","proceeding":null,"authors":["Ilya Zisman","Vladislav Kurenkov","Alexander Nikulin","Viacheslav Sinii","Sergey Kolesnikov"],"abstract":"Recently, extensive studies in Reinforcement Learning have been carried out on the ability of transformers to adapt in-context to various environments and tasks. Current in-context RL methods are limited by their strict requirements for data, which needs to be generated by RL agents or labeled with actions from an optimal policy. In order to address this prevalent problem, we propose AD$^\\varepsilon$, a new data acquisition approach that enables in-context Reinforcement Learning from noise-induced curriculum. We show that it is viable to construct a synthetic noise injection curriculum which helps to obtain learning histories. Moreover, we experimentally demonstrate that it is possible to alleviate the need for generation using optimal policies, with in-context RL still able to outperform the best suboptimal policy in a learning dataset by a 2x margin.","url_abs":"https://arxiv.org/abs/2312.12275v3","url_pdf":"https://arxiv.org/pdf/2312.12275v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"emergence-of-in-context-reinforcement","repo_url":"https://github.com/corl-team/ad-eps","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":null,"task_name":"In-Context Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2312.12275","atlas_url":"https://app.syntology.ai/?focus=2312.12275","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.12275"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/corl-team/ad-eps","reach":null}],"summary":{"ran":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"628df4488d74346e","entry":"ADTransformer","repo":"corl-team/ad-eps","repo_kind":"official","path":"dark_room/ad_darkroom.py","file_url":"https://github.com/corl-team/ad-eps/blob/HEAD/dark_room/ad_darkroom.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"628df4488d74346e"}},{"code_sha256_prefix":"6b539209b4497570","entry":"PositionalEncoding","repo":"corl-team/ad-eps","repo_kind":"official","path":"dark_room/ad_darkroom.py","file_url":"https://github.com/corl-team/ad-eps/blob/HEAD/dark_room/ad_darkroom.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6b539209b4497570"}},{"code_sha256_prefix":"0797809d36877895","entry":"TransformerBlock","repo":"corl-team/ad-eps","repo_kind":"official","path":"dark_room/ad_darkroom.py","file_url":"https://github.com/corl-team/ad-eps/blob/HEAD/dark_room/ad_darkroom.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0797809d36877895"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}