{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/minedojo-building-open-ended-embodied-agents","title":"MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge","arxiv_id":"2206.08853","date":"2022-06-17","proceeding":null,"authors":["Linxi Fan","Guanzhi Wang","Yunfan Jiang","Ajay Mandlekar","Yuncong Yang","Haoyi Zhu","Andrew Tang","De-An Huang","Yuke Zhu","Anima Anandkumar"],"abstract":"Autonomous agents have made great strides in specialist domains like Atari games and Go. However, they typically learn tabula rasa in isolated environments with limited and manually conceived objectives, thus failing to generalize across a wide spectrum of tasks and capabilities. Inspired by how humans continually learn and adapt in the open world, we advocate a trinity of ingredients for building generalist agents: 1) an environment that supports a multitude of tasks and goals, 2) a large-scale database of multimodal knowledge, and 3) a flexible and scalable agent architecture. We introduce MineDojo, a new framework built on the popular Minecraft game that features a simulation suite with thousands of diverse open-ended tasks and an internet-scale knowledge base with Minecraft videos, tutorials, wiki pages, and forum discussions. Using MineDojo's data, we propose a novel agent learning algorithm that leverages large pre-trained video-language models as a learned reward function. Our agent is able to solve a variety of open-ended tasks specified in free-form language without any manually designed dense shaping reward. We open-source the simulation suite, knowledge bases, algorithm implementation, and pretrained models (https://minedojo.org) to promote research towards the goal of generally capable embodied agents.","url_abs":"https://arxiv.org/abs/2206.08853v2","url_pdf":"https://arxiv.org/pdf/2206.08853v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"minedojo-building-open-ended-embodied-agents","repo_url":"https://github.com/MineDojo/MineDojo","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"minedojo-building-open-ended-embodied-agents","repo_url":"https://github.com/AsWali/MineAgentFinal","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"minedojo-building-open-ended-embodied-agents","repo_url":"https://github.com/pku-rl/copl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"minecraft","task_name":"Minecraft"}],"methods":[{"method_slug":"base","method_name":"BASE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2206.08853","atlas_url":"https://app.syntology.ai/?focus=2206.08853","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.08853"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/MineDojo/MineDojo","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pku-rl/copl","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/AsWali/MineAgentFinal","reach":{"status":"ok"}}],"summary":{"ran":7,"ran_draft_wrong":2,"ran_violates":3,"ran_fixture":2,"unverified":14},"by_repo_kind":{"official":{"samples":8,"ran":5,"repositories":1},"listed":{"samples":20,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ad104c89d024fd04","entry":"Buffer","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ad104c89d024fd04"}},{"code_sha256_prefix":"327090e4d81e9be1","entry":"RunningMeanStd","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"327090e4d81e9be1"}},{"code_sha256_prefix":"b6b63db4d56c7e85","entry":"_assign","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b6b63db4d56c7e85"}},{"code_sha256_prefix":"e5f4ff450423f4f7","entry":"_cast","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e5f4ff450423f4f7"}},{"code_sha256_prefix":"934556c7add432d6","entry":"calculate_gae","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"934556c7add432d6"}},{"code_sha256_prefix":"882ec4baf77a7af4","entry":"decode_item_maybe_with_metadata","repo":"MineDojo/MineDojo","repo_kind":"official","path":"minedojo/sim/handlers/util.py","file_url":"https://github.com/MineDojo/MineDojo/blob/HEAD/minedojo/sim/handlers/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"882ec4baf77a7af4"}},{"code_sha256_prefix":"6af25de29c5f7124","entry":"encode_item_with_metadata","repo":"MineDojo/MineDojo","repo_kind":"official","path":"minedojo/sim/handlers/util.py","file_url":"https://github.com/MineDojo/MineDojo/blob/HEAD/minedojo/sim/handlers/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6af25de29c5f7124"}},{"code_sha256_prefix":"07902b0c59d65132","entry":"explained_variance","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"07902b0c59d65132"}},{"code_sha256_prefix":"cfa55d08613e2d96","entry":"get_callable","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cfa55d08613e2d96"}},{"code_sha256_prefix":"31090e4cf12c6ba1","entry":"get_key_from_id","repo":"MineDojo/MineDojo","repo_kind":"official","path":"minedojo/sim/mc_meta/mc.py","file_url":"https://github.com/MineDojo/MineDojo/blob/HEAD/minedojo/sim/mc_meta/mc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"31090e4cf12c6ba1"}},{"code_sha256_prefix":"44576799efab9213","entry":"item_list_contains","repo":"MineDojo/MineDojo","repo_kind":"official","path":"minedojo/sim/handlers/util.py","file_url":"https://github.com/MineDojo/MineDojo/blob/HEAD/minedojo/sim/handlers/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"44576799efab9213"}},{"code_sha256_prefix":"623c17e586414975","entry":"map_slot_number_to_cmd_slot","repo":"MineDojo/MineDojo","repo_kind":"official","path":"minedojo/sim/inventory.py","file_url":"https://github.com/MineDojo/MineDojo/blob/HEAD/minedojo/sim/inventory.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"623c17e586414975"}},{"code_sha256_prefix":"db5606ac4557835f","entry":"preprocess","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"db5606ac4557835f"}},{"code_sha256_prefix":"eb736d83f82d921a","entry":"stack_obs","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eb736d83f82d921a"}},{"code_sha256_prefix":"7afc0befd6b15a83","entry":"Actor","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7afc0befd6b15a83"}},{"code_sha256_prefix":"8051c6b988c6c698","entry":"CLIPReward","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8051c6b988c6c698"}},{"code_sha256_prefix":"e9c9518cae5dc1e0","entry":"Learner","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e9c9518cae5dc1e0"}},{"code_sha256_prefix":"cf96afb55e80a2b0","entry":"ModifiedPPOWorker","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cf96afb55e80a2b0"}},{"code_sha256_prefix":"62ea7f1fcfd84736","entry":"PPOActor","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"62ea7f1fcfd84736"}},{"code_sha256_prefix":"f6f0eaea0b8e2bdd","entry":"PPOLearner","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f6f0eaea0b8e2bdd"}},{"code_sha256_prefix":"4b0d0097d64ea914","entry":"PPOWorker","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4b0d0097d64ea914"}},{"code_sha256_prefix":"21a5ad45e61012fb","entry":"assign","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"21a5ad45e61012fb"}},{"code_sha256_prefix":"fd5ba1529b4c89b6","entry":"get_inventory_vector","repo":"MineDojo/MineDojo","repo_kind":"official","path":"minedojo/sim/wrappers/utils.py","file_url":"https://github.com/MineDojo/MineDojo/blob/HEAD/minedojo/sim/wrappers/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fd5ba1529b4c89b6"}},{"code_sha256_prefix":"68eacb4c0254a0c4","entry":"get_item_id","repo":"MineDojo/MineDojo","repo_kind":"official","path":"minedojo/sim/mc_meta/mc.py","file_url":"https://github.com/MineDojo/MineDojo/blob/HEAD/minedojo/sim/mc_meta/mc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"68eacb4c0254a0c4"}},{"code_sha256_prefix":"cf26db7df2302923","entry":"get_scheduler","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cf26db7df2302923"}},{"code_sha256_prefix":"f5cbe0699dd7d077","entry":"multimap","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f5cbe0699dd7d077"}},{"code_sha256_prefix":"6506006d68a080bb","entry":"recursive_dict_eq","repo":"MineDojo/MineDojo","repo_kind":"official","path":"minedojo/sim/mc_meta/mc.py","file_url":"https://github.com/MineDojo/MineDojo/blob/HEAD/minedojo/sim/mc_meta/mc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6506006d68a080bb"}},{"code_sha256_prefix":"d7084f29067204ad","entry":"worker_class","repo":"pku-rl/copl","repo_kind":"listed","path":"src/core/ppo.py","file_url":"https://github.com/pku-rl/copl/blob/HEAD/src/core/ppo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d7084f29067204ad"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}