{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/lemur-harmonizing-natural-language-and-code","title":"Lemur: Harmonizing Natural Language and Code for Language Agents","arxiv_id":"2310.06830","date":"2023-10-10","proceeding":null,"authors":["Yiheng Xu","Hongjin Su","Chen Xing","Boyu Mi","Qian Liu","Weijia Shi","Binyuan Hui","Fan Zhou","Yitao Liu","Tianbao Xie","Zhoujun Cheng","Siheng Zhao","Lingpeng Kong","Bailin Wang","Caiming Xiong","Tao Yu"],"abstract":"We introduce Lemur and Lemur-Chat, openly accessible language models optimized for both natural language and coding capabilities to serve as the backbone of versatile language agents. The evolution from language chat models to functional language agents demands that models not only master human interaction, reasoning, and planning but also ensure grounding in the relevant environments. This calls for a harmonious blend of language and coding capabilities in the models. Lemur and Lemur-Chat are proposed to address this necessity, demonstrating balanced proficiencies in both domains, unlike existing open-source models that tend to specialize in either. Through meticulous pre-training using a code-intensive corpus and instruction fine-tuning on text and code data, our models achieve state-of-the-art averaged performance across diverse text and coding benchmarks among open-source models. Comprehensive experiments demonstrate Lemur's superiority over existing open-source models and its proficiency across various agent tasks involving human communication, tool usage, and interaction under fully- and partially- observable environments. The harmonization between natural and programming languages enables Lemur-Chat to significantly narrow the gap with proprietary models on agent abilities, providing key insights into developing advanced open-source agents adept at reasoning, planning, and operating seamlessly across environments. https://github.com/OpenLemur/Lemur","url_abs":"https://arxiv.org/abs/2310.06830v2","url_pdf":"https://arxiv.org/pdf/2310.06830v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"lemur-harmonizing-natural-language-and-code","repo_url":"https://github.com/openlemur/lemur","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.06830","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.06830"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/OpenLemur/Lemur","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/openlemur/lemur","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":6,"unverified":3},"by_repo_kind":{"official":{"samples":9,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8140036b8f8671e0","entry":"dispatch_openai_chat_requesets","repo":"OpenLemur/Lemur","repo_kind":"official","path":"xchat/eval/dispatch_openai_requests.py","file_url":"https://github.com/OpenLemur/Lemur/blob/HEAD/xchat/eval/dispatch_openai_requests.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8140036b8f8671e0"}},{"code_sha256_prefix":"1dbfc8b9e115e56e","entry":"dispatch_openai_prompt_requesets","repo":"OpenLemur/Lemur","repo_kind":"official","path":"xchat/eval/dispatch_openai_requests.py","file_url":"https://github.com/OpenLemur/Lemur/blob/HEAD/xchat/eval/dispatch_openai_requests.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1dbfc8b9e115e56e"}},{"code_sha256_prefix":"af79b5e511f124e0","entry":"estimate_pass_at_k","repo":"OpenLemur/Lemur","repo_kind":"official","path":"xchat/eval/mbpp/evaluation.py","file_url":"https://github.com/OpenLemur/Lemur/blob/HEAD/xchat/eval/mbpp/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"af79b5e511f124e0"}},{"code_sha256_prefix":"3eeddf88871321ab","entry":"generate_completions","repo":"OpenLemur/Lemur","repo_kind":"official","path":"xchat/eval/utils.py","file_url":"https://github.com/OpenLemur/Lemur/blob/HEAD/xchat/eval/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3eeddf88871321ab"}},{"code_sha256_prefix":"98257d49f14f4875","entry":"getprompts","repo":"OpenLemur/Lemur","repo_kind":"official","path":"xchat/eval/humaneval/prompts.py","file_url":"https://github.com/OpenLemur/Lemur/blob/HEAD/xchat/eval/humaneval/prompts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"98257d49f14f4875"}},{"code_sha256_prefix":"1cf1721a0bcb876e","entry":"read_problems","repo":"OpenLemur/Lemur","repo_kind":"official","path":"xchat/eval/mbpp/data.py","file_url":"https://github.com/OpenLemur/Lemur/blob/HEAD/xchat/eval/mbpp/data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1cf1721a0bcb876e"}},{"code_sha256_prefix":"b8cf46548e4cd074","entry":"check_correctness","repo":"OpenLemur/Lemur","repo_kind":"official","path":"xchat/eval/mbpp/execution.py","file_url":"https://github.com/OpenLemur/Lemur/blob/HEAD/xchat/eval/mbpp/execution.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b8cf46548e4cd074"}},{"code_sha256_prefix":"19dce514835bd816","entry":"get_next_word_predictions","repo":"OpenLemur/Lemur","repo_kind":"official","path":"xchat/eval/utils.py","file_url":"https://github.com/OpenLemur/Lemur/blob/HEAD/xchat/eval/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"19dce514835bd816"}},{"code_sha256_prefix":"be8165ca7e91d372","entry":"load_hf_lm_and_tokenizer","repo":"OpenLemur/Lemur","repo_kind":"official","path":"xchat/eval/utils.py","file_url":"https://github.com/OpenLemur/Lemur/blob/HEAD/xchat/eval/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"be8165ca7e91d372"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}