{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/teacher-algorithms-for-curriculum-learning-of","title":"Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments","arxiv_id":"1910.07224","date":"2019-10-16","proceeding":null,"authors":["Rémy Portelas","Cédric Colas","Katja Hofmann","Pierre-Yves Oudeyer"],"abstract":"We consider the problem of how a teacher algorithm can enable an unknown Deep Reinforcement Learning (DRL) student to become good at a skill over a wide range of diverse environments. To do so, we study how a teacher algorithm can learn to generate a learning curriculum, whereby it sequentially samples parameters controlling a stochastic procedural generation of environments. Because it does not initially know the capacities of its student, a key challenge for the teacher is to discover which environments are easy, difficult or unlearnable, and in what order to propose them to maximize the efficiency of learning over the learnable ones. To achieve this, this problem is transformed into a surrogate continuous bandit problem where the teacher samples environments in order to maximize absolute learning progress of its student. We present a new algorithm modeling absolute learning progress with Gaussian mixture models (ALP-GMM). We also adapt existing algorithms and provide a complete study in the context of DRL. Using parameterized variants of the BipedalWalker environment, we study their efficiency to personalize a learning curriculum for different learners (embodiments), their robustness to the ratio of learnable/unlearnable environments, and their scalability to non-linear and high-dimensional parameter spaces. Videos and code are available at https://github.com/flowersteam/teachDeepRL.","url_abs":"https://arxiv.org/abs/1910.07224v1","url_pdf":"https://arxiv.org/pdf/1910.07224v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"teacher-algorithms-for-curriculum-learning-of","repo_url":"https://github.com/flowersteam/teachDeepRL","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"teacher-algorithms-for-curriculum-learning-of","repo_url":"https://github.com/facebookresearch/dcd","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"}],"methods":[{"method_slug":"alp-gmm","method_name":"ALP-GMM"}],"datasets_introduced":[],"methods_introduced":[{"slug":"alp-gmm","name":"ALP-GMM","full_name":"Absolute Learning Progress and Gaussian Mixture Models for Automatic Curriculum Learning"}],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1910.07224","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1910.07224"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/facebookresearch/dcd","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/flowersteam/teachDeepRL","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":7},"by_repo_kind":{"official":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"06521a4018d88f64","entry":"Rotate2D","repo":"flowersteam/teachDeepRL","repo_kind":"official","path":"teachDRL/gym_flowers/envs/bipedal_walker_continuous.py","file_url":"https://github.com/flowersteam/teachDeepRL/blob/HEAD/teachDRL/gym_flowers/envs/bipedal_walker_continuous.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"06521a4018d88f64"}},{"code_sha256_prefix":"a2580d7c4a763348","entry":"get_vars","repo":"flowersteam/teachDeepRL","repo_kind":"official","path":"teachDRL/spinup/algos/ddpg/core.py","file_url":"https://github.com/flowersteam/teachDeepRL/blob/HEAD/teachDRL/spinup/algos/ddpg/core.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a2580d7c4a763348"}},{"code_sha256_prefix":"e68b07bcdded00fa","entry":"mlp","repo":"flowersteam/teachDeepRL","repo_kind":"official","path":"teachDRL/spinup/algos/ddpg/core.py","file_url":"https://github.com/flowersteam/teachDeepRL/blob/HEAD/teachDRL/spinup/algos/ddpg/core.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e68b07bcdded00fa"}},{"code_sha256_prefix":"a80e3eef65a4a0a4","entry":"param_dict_to_param_vec","repo":"flowersteam/teachDeepRL","repo_kind":"official","path":"teachDRL/teachers/teacher_controller.py","file_url":"https://github.com/flowersteam/teachDeepRL/blob/HEAD/teachDRL/teachers/teacher_controller.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a80e3eef65a4a0a4"}},{"code_sha256_prefix":"30f260501f2782b6","entry":"param_vec_to_param_dict","repo":"flowersteam/teachDeepRL","repo_kind":"official","path":"teachDRL/teachers/teacher_controller.py","file_url":"https://github.com/flowersteam/teachDeepRL/blob/HEAD/teachDRL/teachers/teacher_controller.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"30f260501f2782b6"}},{"code_sha256_prefix":"17b03c462ce92576","entry":"placeholder","repo":"flowersteam/teachDeepRL","repo_kind":"official","path":"teachDRL/spinup/algos/ddpg/core.py","file_url":"https://github.com/flowersteam/teachDeepRL/blob/HEAD/teachDRL/spinup/algos/ddpg/core.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"17b03c462ce92576"}},{"code_sha256_prefix":"762e08fee0c22ebd","entry":"proportional_choice","repo":"flowersteam/teachDeepRL","repo_kind":"official","path":"teachDRL/teachers/algos/alp_gmm.py","file_url":"https://github.com/flowersteam/teachDeepRL/blob/HEAD/teachDRL/teachers/algos/alp_gmm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"762e08fee0c22ebd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}