{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/small-llms-are-weak-tool-learners-a-multi-llm","title":"Small LLMs Are Weak Tool Learners: A Multi-LLM Agent","arxiv_id":"2401.07324","date":"2024-01-14","proceeding":null,"authors":["Weizhou Shen","Chenliang Li","Hongzhan Chen","Ming Yan","Xiaojun Quan","Hehong Chen","Ji Zhang","Fei Huang"],"abstract":"Large Language Model (LLM) agents significantly extend the capabilities of standalone LLMs, empowering them to interact with external tools (e.g., APIs, functions) and complete various tasks in a self-directed fashion. The challenge of tool use demands that LLMs not only understand user queries and generate answers accurately but also excel in task planning, tool invocation, and result summarization. While traditional works focus on training a single LLM with all these capabilities, performance limitations become apparent, particularly with smaller models. To overcome these challenges, we propose a novel approach that decomposes the aforementioned capabilities into a planner, caller, and summarizer. Each component is implemented by a single LLM that focuses on a specific capability and collaborates with others to accomplish the task. This modular framework facilitates individual updates and the potential use of smaller LLMs for building each capability. To effectively train this framework, we introduce a two-stage training paradigm. First, we fine-tune a backbone LLM on the entire dataset without discriminating sub-tasks, providing the model with a comprehensive understanding of the task. Second, the fine-tuned LLM is used to instantiate the planner, caller, and summarizer respectively, which are continually fine-tuned on respective sub-tasks. Evaluation across various tool-use benchmarks illustrates that our proposed multi-LLM framework surpasses the traditional single-LLM approach, highlighting its efficacy and advantages in tool learning.","url_abs":"https://arxiv.org/abs/2401.07324v3","url_pdf":"https://arxiv.org/pdf/2401.07324v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"small-llms-are-weak-tool-learners-a-multi-llm","repo_url":"https://github.com/x-plug/multi-llm-agent","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"large-language-model","task_name":"Large Language Model"},{"task_slug":"task-planning","task_name":"Task Planning"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2401.07324","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.07324"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/x-plug/multi-llm-agent","reach":null}],"summary":{"ran":1,"ran_draft_wrong":3,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"bed576a0bbc5aca7","entry":"SimpleChatIO","repo":"x-plug/multi-llm-agent","repo_kind":"official","path":"ToolBench-multiLLM/toolbench/inference/LLM/collab_agent_model.py","file_url":"https://github.com/x-plug/multi-llm-agent/blob/HEAD/ToolBench-multiLLM/toolbench/inference/LLM/collab_agent_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bed576a0bbc5aca7"}},{"code_sha256_prefix":"b0051984c25d68c5","entry":"collab_agent_parser","repo":"x-plug/multi-llm-agent","repo_kind":"official","path":"ToolBench-multiLLM/toolbench/inference/LLM/collab_agent_model.py","file_url":"https://github.com/x-plug/multi-llm-agent/blob/HEAD/ToolBench-multiLLM/toolbench/inference/LLM/collab_agent_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b0051984c25d68c5"}},{"code_sha256_prefix":"67d55c8e9077d9e7","entry":"generate_stream","repo":"x-plug/multi-llm-agent","repo_kind":"official","path":"ToolBench-multiLLM/toolbench/inference/LLM/collab_agent_model.py","file_url":"https://github.com/x-plug/multi-llm-agent/blob/HEAD/ToolBench-multiLLM/toolbench/inference/LLM/collab_agent_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"67d55c8e9077d9e7"}},{"code_sha256_prefix":"93c25411cf60d992","entry":"prepare_logits_processor","repo":"x-plug/multi-llm-agent","repo_kind":"official","path":"ToolBench-multiLLM/toolbench/inference/LLM/collab_agent_model.py","file_url":"https://github.com/x-plug/multi-llm-agent/blob/HEAD/ToolBench-multiLLM/toolbench/inference/LLM/collab_agent_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"93c25411cf60d992"}},{"code_sha256_prefix":"3d81fd3240dba079","entry":"ChatIO","repo":"x-plug/multi-llm-agent","repo_kind":"official","path":"ToolBench-multiLLM/toolbench/inference/LLM/collab_agent_model.py","file_url":"https://github.com/x-plug/multi-llm-agent/blob/HEAD/ToolBench-multiLLM/toolbench/inference/LLM/collab_agent_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3d81fd3240dba079"}},{"code_sha256_prefix":"52618fa24b988a40","entry":"CollbaAgentV3","repo":"x-plug/multi-llm-agent","repo_kind":"official","path":"ToolBench-multiLLM/toolbench/inference/LLM/collab_agent_model.py","file_url":"https://github.com/x-plug/multi-llm-agent/blob/HEAD/ToolBench-multiLLM/toolbench/inference/LLM/collab_agent_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"52618fa24b988a40"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}