{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/thought-cloning-learning-to-think-while-1","title":"Thought Cloning: Learning to Think while Acting by Imitating Human Thinking","arxiv_id":"2306.00323","date":"2023-06-01","proceeding":"NeurIPS 2023 11","authors":["Shengran Hu","Jeff Clune"],"abstract":"Language is often considered a key aspect of human thinking, providing us with exceptional abilities to generalize, explore, plan, replan, and adapt to new situations. However, Reinforcement Learning (RL) agents are far from human-level performance in any of these abilities. We hypothesize one reason for such cognitive deficiencies is that they lack the benefits of thinking in language and that we can improve AI agents by training them to think like humans do. We introduce a novel Imitation Learning framework, Thought Cloning, where the idea is to not just clone the behaviors of human demonstrators, but also the thoughts humans have as they perform these behaviors. While we expect Thought Cloning to truly shine at scale on internet-sized datasets of humans thinking out loud while acting (e.g. online videos with transcripts), here we conduct experiments in a domain where the thinking and action data are synthetically generated. Results reveal that Thought Cloning learns much faster than Behavioral Cloning and its performance advantage grows the further out of distribution test tasks are, highlighting its ability to better handle novel situations. Thought Cloning also provides important benefits for AI Safety and Interpretability, and makes it easier to debug and improve AI. Because we can observe the agent's thoughts, we can (1) more easily diagnose why things are going wrong, making it easier to fix the problem, (2) steer the agent by correcting its thinking, or (3) prevent it from doing unsafe things it plans to do. Overall, by training agents how to think as well as behave, Thought Cloning creates safer, more powerful agents.","url_abs":"https://arxiv.org/abs/2306.00323v3","url_pdf":"https://arxiv.org/pdf/2306.00323v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"thought-cloning-learning-to-think-while-1","repo_url":"https://github.com/ShengranHu/Thought-Cloning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"imitation-learning","task_name":"Imitation Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.00323","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.00323"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ShengranHu/Thought-Cloning","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":7},"by_repo_kind":{"official":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5ddbfe9ab59cd4a4","entry":"evaluate","repo":"ShengranHu/Thought-Cloning","repo_kind":"official","path":"babyai/evaluate.py","file_url":"https://github.com/ShengranHu/Thought-Cloning/blob/HEAD/babyai/evaluate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5ddbfe9ab59cd4a4"}},{"code_sha256_prefix":"3a317898c18a2a4e","entry":"evaluate_demo_agent","repo":"ShengranHu/Thought-Cloning","repo_kind":"official","path":"babyai/evaluate.py","file_url":"https://github.com/ShengranHu/Thought-Cloning/blob/HEAD/babyai/evaluate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3a317898c18a2a4e"}},{"code_sha256_prefix":"2cbc87bf930aff09","entry":"get_model_dir","repo":"ShengranHu/Thought-Cloning","repo_kind":"official","path":"babyai/utils/model.py","file_url":"https://github.com/ShengranHu/Thought-Cloning/blob/HEAD/babyai/utils/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2cbc87bf930aff09"}},{"code_sha256_prefix":"5e7bbe8728391df2","entry":"get_model_path","repo":"ShengranHu/Thought-Cloning","repo_kind":"official","path":"babyai/utils/model.py","file_url":"https://github.com/ShengranHu/Thought-Cloning/blob/HEAD/babyai/utils/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5e7bbe8728391df2"}},{"code_sha256_prefix":"63cced98b78ed47a","entry":"load_model","repo":"ShengranHu/Thought-Cloning","repo_kind":"official","path":"babyai/utils/model.py","file_url":"https://github.com/ShengranHu/Thought-Cloning/blob/HEAD/babyai/utils/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"63cced98b78ed47a"}},{"code_sha256_prefix":"c1ffbae9fb9c6fc2","entry":"manhattan_distance","repo":"ShengranHu/Thought-Cloning","repo_kind":"official","path":"babyai/bot.py","file_url":"https://github.com/ShengranHu/Thought-Cloning/blob/HEAD/babyai/bot.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c1ffbae9fb9c6fc2"}},{"code_sha256_prefix":"bb2e81ff7543e52a","entry":"maskedNll","repo":"ShengranHu/Thought-Cloning","repo_kind":"official","path":"babyai/submodules.py","file_url":"https://github.com/ShengranHu/Thought-Cloning/blob/HEAD/babyai/submodules.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bb2e81ff7543e52a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}