{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/agent-laboratory-using-llm-agents-as-research","title":"Agent Laboratory: Using LLM Agents as Research Assistants","arxiv_id":"2501.04227","date":"2025-01-08","proceeding":null,"authors":["Samuel Schmidgall","Yusheng Su","Ze Wang","Ximeng Sun","Jialian Wu","Xiaodong Yu","Jiang Liu","Michael Moor","Zicheng Liu","Emad Barsoum"],"abstract":"Historically, scientific discovery has been a lengthy and costly process, demanding substantial time and resources from initial conception to final results. To accelerate scientific discovery, reduce research costs, and improve research quality, we introduce Agent Laboratory, an autonomous LLM-based framework capable of completing the entire research process. This framework accepts a human-provided research idea and progresses through three stages--literature review, experimentation, and report writing to produce comprehensive research outputs, including a code repository and a research report, while enabling users to provide feedback and guidance at each stage. We deploy Agent Laboratory with various state-of-the-art LLMs and invite multiple researchers to assess its quality by participating in a survey, providing human feedback to guide the research process, and then evaluate the final paper. We found that: (1) Agent Laboratory driven by o1-preview generates the best research outcomes; (2) The generated machine learning code is able to achieve state-of-the-art performance compared to existing methods; (3) Human involvement, providing feedback at each stage, significantly improves the overall quality of research; (4) Agent Laboratory significantly reduces research expenses, achieving an 84% decrease compared to previous autonomous research methods. We hope Agent Laboratory enables researchers to allocate more effort toward creative ideation rather than low-level coding and writing, ultimately accelerating scientific discovery.","url_abs":"https://arxiv.org/abs/2501.04227v2","url_pdf":"https://arxiv.org/pdf/2501.04227v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"agent-laboratory-using-llm-agents-as-research","repo_url":"https://github.com/samuelschmidgall/agentlaboratory","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"agent-laboratory-using-llm-agents-as-research","repo_url":"https://github.com/Masao-Taketani/LocalAgentLaboratory","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"scientific-discovery","task_name":"scientific discovery"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2501.04227","atlas_url":"https://app.syntology.ai/?focus=2501.04227","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2501.04227"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/samuelschmidgall/agentlaboratory","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Masao-Taketani/LocalAgentLaboratory","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"unverified":4},"by_repo_kind":{"listed":{"samples":9,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"67f528bb583ccdd6","entry":"compile_latex","repo":"Masao-Taketani/LocalAgentLaboratory","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/Masao-Taketani/LocalAgentLaboratory/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"67f528bb583ccdd6"}},{"code_sha256_prefix":"9f6ad073cc70700d","entry":"execute_code","repo":"Masao-Taketani/LocalAgentLaboratory","repo_kind":"listed","path":"tools.py","file_url":"https://github.com/Masao-Taketani/LocalAgentLaboratory/blob/HEAD/tools.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f6ad073cc70700d"}},{"code_sha256_prefix":"495eb30d4561e646","entry":"extract_json_between_markers","repo":"Masao-Taketani/LocalAgentLaboratory","repo_kind":"listed","path":"agents.py","file_url":"https://github.com/Masao-Taketani/LocalAgentLaboratory/blob/HEAD/agents.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"495eb30d4561e646"}},{"code_sha256_prefix":"5d7e8ba88a1d4f7f","entry":"process_think_model_output","repo":"Masao-Taketani/LocalAgentLaboratory","repo_kind":"listed","path":"inference.py","file_url":"https://github.com/Masao-Taketani/LocalAgentLaboratory/blob/HEAD/inference.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5d7e8ba88a1d4f7f"}},{"code_sha256_prefix":"528c51546feae850","entry":"query_model","repo":"Masao-Taketani/LocalAgentLaboratory","repo_kind":"listed","path":"inference.py","file_url":"https://github.com/Masao-Taketani/LocalAgentLaboratory/blob/HEAD/inference.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"528c51546feae850"}},{"code_sha256_prefix":"54c2e719cebe9b7b","entry":"adjust_phase_status","repo":"Masao-Taketani/LocalAgentLaboratory","repo_kind":"listed","path":"ai_lab_repo.py","file_url":"https://github.com/Masao-Taketani/LocalAgentLaboratory/blob/HEAD/ai_lab_repo.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"54c2e719cebe9b7b"}},{"code_sha256_prefix":"8459a0a1d17892b1","entry":"clip_tokens","repo":"Masao-Taketani/LocalAgentLaboratory","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/Masao-Taketani/LocalAgentLaboratory/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8459a0a1d17892b1"}},{"code_sha256_prefix":"023b788084b451b0","entry":"count_tokens","repo":"Masao-Taketani/LocalAgentLaboratory","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/Masao-Taketani/LocalAgentLaboratory/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"023b788084b451b0"}},{"code_sha256_prefix":"7dfdf3c5c19052b6","entry":"init_hf_pipe","repo":"Masao-Taketani/LocalAgentLaboratory","repo_kind":"listed","path":"agents.py","file_url":"https://github.com/Masao-Taketani/LocalAgentLaboratory/blob/HEAD/agents.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7dfdf3c5c19052b6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}