{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/traced-transition-aware-regret-approximation","title":"TRACED: Transition-aware Regret Approximation with Co-learnability for Environment Design","arxiv_id":"2506.19997","date":"2025-06-24","proceeding":null,"authors":["Geonwoo Cho","Jaegyun Im","JIhwan Lee","Hojun Yi","Sejin Kim","Sundong Kim"],"abstract":"Generalizing deep reinforcement learning agents to unseen environments remains a significant challenge. One promising solution is Unsupervised Environment Design (UED), a co-evolutionary framework in which a teacher adaptively generates tasks with high learning potential, while a student learns a robust policy from this evolving curriculum. Existing UED methods typically measure learning potential via regret, the gap between optimal and current performance, approximated solely by value-function loss. Building on these approaches, we introduce the transition prediction error as an additional term in our regret approximation. To capture how training on one task affects performance on others, we further propose a lightweight metric called co-learnability. By combining these two measures, we present Transition-aware Regret Approximation with Co-learnability for Environment Design (TRACED). Empirical evaluations show that TRACED yields curricula that improve zero-shot generalization across multiple benchmarks while requiring up to 2x fewer environment interactions than strong baselines. Ablation studies confirm that the transition prediction error drives rapid complexity ramp-up and that co-learnability delivers additional gains when paired with the transition prediction error. These results demonstrate how refined regret approximation and explicit modeling of task relationships can be leveraged for sample-efficient curriculum design in UED.","url_abs":"https://arxiv.org/abs/2506.19997v1","url_pdf":"https://arxiv.org/pdf/2506.19997v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"traced-transition-aware-regret-approximation","repo_url":"https://github.com/cho-geonwoo/traced","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"zero-shot-generalization","task_name":"Zero-shot Generalization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2506.19997","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.19997"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Cho-Geonwoo/TRACED","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cho-geonwoo/traced","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_draft_wrong":1,"unverified":5},"by_repo_kind":{"official":{"samples":6,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"9de355e93051e4ad","entry":"init","repo":"Cho-Geonwoo/TRACED","repo_kind":"official","path":"models/recurrent_walker_models.py","file_url":"https://github.com/Cho-Geonwoo/TRACED/blob/HEAD/models/recurrent_walker_models.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9de355e93051e4ad"}},{"code_sha256_prefix":"f9f894531a94e423","entry":"apply_permutation","repo":"Cho-Geonwoo/TRACED","repo_kind":"official","path":"models/rnn.py","file_url":"https://github.com/Cho-Geonwoo/TRACED/blob/HEAD/models/rnn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f9f894531a94e423"}},{"code_sha256_prefix":"f8db408bc65dbb3a","entry":"init","repo":"Cho-Geonwoo/TRACED","repo_kind":"official","path":"models/common.py","file_url":"https://github.com/Cho-Geonwoo/TRACED/blob/HEAD/models/common.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f8db408bc65dbb3a"}},{"code_sha256_prefix":"759061064b74bf09","entry":"make_fc_layers_with_hidden_sizes","repo":"Cho-Geonwoo/TRACED","repo_kind":"official","path":"models/common.py","file_url":"https://github.com/Cho-Geonwoo/TRACED/blob/HEAD/models/common.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"759061064b74bf09"}},{"code_sha256_prefix":"60463321ab6527df","entry":"one_hot","repo":"Cho-Geonwoo/TRACED","repo_kind":"official","path":"models/common.py","file_url":"https://github.com/Cho-Geonwoo/TRACED/blob/HEAD/models/common.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"60463321ab6527df"}},{"code_sha256_prefix":"686bf9e39d5047a9","entry":"to_tensor","repo":"Cho-Geonwoo/TRACED","repo_kind":"official","path":"algos/storage.py","file_url":"https://github.com/Cho-Geonwoo/TRACED/blob/HEAD/algos/storage.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"686bf9e39d5047a9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}