{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/combining-learning-from-human-feedback-and","title":"Combining Learning from Human Feedback and Knowledge Engineering to Solve Hierarchical Tasks in Minecraft","arxiv_id":"2112.03482","date":"2021-12-07","proceeding":null,"authors":["Vinicius G. Goecks","Nicholas Waytowich","David Watkins-Valls","Bharat Prakash"],"abstract":"Real-world tasks of interest are generally poorly defined by human-readable descriptions and have no pre-defined reward signals unless it is defined by a human designer. Conversely, data-driven algorithms are often designed to solve a specific, narrowly defined, task with performance metrics that drives the agent's learning. In this work, we present the solution that won first place and was awarded the most human-like agent in the 2021 NeurIPS Competition MineRL BASALT Challenge: Learning from Human Feedback in Minecraft, which challenged participants to use human data to solve four tasks defined only by a natural language description and no reward function. Our approach uses the available human demonstration data to train an imitation learning policy for navigation and additional human feedback to train an image classifier. These modules, combined with an estimated odometry map, become a powerful state-machine designed to utilize human knowledge in a natural hierarchical paradigm. We compare this hybrid intelligence approach to both end-to-end machine learning and pure engineered solutions, which are then judged by human evaluators. Codebase is available at https://github.com/viniciusguigo/kairos_minerl_basalt.","url_abs":"https://arxiv.org/abs/2112.03482v2","url_pdf":"https://arxiv.org/pdf/2112.03482v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"combining-learning-from-human-feedback-and","repo_url":"https://github.com/viniciusguigo/kairos_minerl_basalt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"imitation-learning","task_name":"Imitation Learning"},{"task_slug":"minecraft","task_name":"Minecraft"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2112.03482","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2112.03482"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/viniciusguigo/kairos_minerl_basalt","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":9},"by_repo_kind":{"official":{"samples":9,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"481210e6f5732a32","entry":"compute_accuracy","repo":"viniciusguigo/kairos_minerl_basalt","repo_kind":"official","path":"kairos_minerl/src/kairos_minerl/behavior_cloner.py","file_url":"https://github.com/viniciusguigo/kairos_minerl_basalt/blob/HEAD/kairos_minerl/src/kairos_minerl/behavior_cloner.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"481210e6f5732a32"}},{"code_sha256_prefix":"9fd4e3706a238d72","entry":"compute_accuracy","repo":"viniciusguigo/kairos_minerl_basalt","repo_kind":"official","path":"kairos_minerl/src/kairos_minerl/state_classifier.py","file_url":"https://github.com/viniciusguigo/kairos_minerl_basalt/blob/HEAD/kairos_minerl/src/kairos_minerl/state_classifier.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9fd4e3706a238d72"}},{"code_sha256_prefix":"336562bd9a96072a","entry":"dataset_action_batch_to_actions","repo":"viniciusguigo/kairos_minerl_basalt","repo_kind":"official","path":"basalt_baselines/anssi_bc.py","file_url":"https://github.com/viniciusguigo/kairos_minerl_basalt/blob/HEAD/basalt_baselines/anssi_bc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"336562bd9a96072a"}},{"code_sha256_prefix":"79b2380705eacafd","entry":"load_training_data","repo":"viniciusguigo/kairos_minerl_basalt","repo_kind":"official","path":"kairos_minerl/src/kairos_minerl/behavior_cloner.py","file_url":"https://github.com/viniciusguigo/kairos_minerl_basalt/blob/HEAD/kairos_minerl/src/kairos_minerl/behavior_cloner.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"79b2380705eacafd"}},{"code_sha256_prefix":"7e9df6464fd9a23c","entry":"load_training_data","repo":"viniciusguigo/kairos_minerl_basalt","repo_kind":"official","path":"kairos_minerl/src/kairos_minerl/state_classifier.py","file_url":"https://github.com/viniciusguigo/kairos_minerl_basalt/blob/HEAD/kairos_minerl/src/kairos_minerl/state_classifier.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7e9df6464fd9a23c"}},{"code_sha256_prefix":"824a62d38a67639d","entry":"load_training_data_old","repo":"viniciusguigo/kairos_minerl_basalt","repo_kind":"official","path":"kairos_minerl/src/kairos_minerl/state_classifier.py","file_url":"https://github.com/viniciusguigo/kairos_minerl_basalt/blob/HEAD/kairos_minerl/src/kairos_minerl/state_classifier.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"824a62d38a67639d"}},{"code_sha256_prefix":"91730314fc5f688e","entry":"processed_actions_to_wrapper_actions_Animalpen","repo":"viniciusguigo/kairos_minerl_basalt","repo_kind":"official","path":"kairos_minerl/src/kairos_minerl/gail_wrapper.py","file_url":"https://github.com/viniciusguigo/kairos_minerl_basalt/blob/HEAD/kairos_minerl/src/kairos_minerl/gail_wrapper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"91730314fc5f688e"}},{"code_sha256_prefix":"e8355439c3ca041d","entry":"processed_actions_to_wrapper_actions_FindCave","repo":"viniciusguigo/kairos_minerl_basalt","repo_kind":"official","path":"kairos_minerl/src/kairos_minerl/gail_wrapper.py","file_url":"https://github.com/viniciusguigo/kairos_minerl_basalt/blob/HEAD/kairos_minerl/src/kairos_minerl/gail_wrapper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e8355439c3ca041d"}},{"code_sha256_prefix":"14b6914a84600356","entry":"processed_actions_to_wrapper_actions_Waterfall","repo":"viniciusguigo/kairos_minerl_basalt","repo_kind":"official","path":"kairos_minerl/src/kairos_minerl/gail_wrapper.py","file_url":"https://github.com/viniciusguigo/kairos_minerl_basalt/blob/HEAD/kairos_minerl/src/kairos_minerl/gail_wrapper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"14b6914a84600356"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}