{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scilitllm-how-to-adapt-llms-for-scientific","title":"SciLitLLM: How to Adapt LLMs for Scientific Literature Understanding","arxiv_id":"2408.15545","date":"2024-08-28","proceeding":null,"authors":["Sihang Li","Jin Huang","Jiaxi Zhuang","Yaorui Shi","Xiaochen Cai","Mingjun Xu","Xiang Wang","Linfeng Zhang","Guolin Ke","Hengxing Cai"],"abstract":"Scientific literature understanding is crucial for extracting targeted information and garnering insights, thereby significantly advancing scientific discovery. Despite the remarkable success of Large Language Models (LLMs), they face challenges in scientific literature understanding, primarily due to (1) a lack of scientific knowledge and (2) unfamiliarity with specialized scientific tasks. To develop an LLM specialized in scientific literature understanding, we propose a hybrid strategy that integrates continual pre-training (CPT) and supervised fine-tuning (SFT), to simultaneously infuse scientific domain knowledge and enhance instruction-following capabilities for domain-specific tasks.cIn this process, we identify two key challenges: (1) constructing high-quality CPT corpora, and (2) generating diverse SFT instructions. We address these challenges through a meticulous pipeline, including PDF text extraction, parsing content error correction, quality filtering, and synthetic instruction creation. Applying this strategy, we present a suite of LLMs: SciLitLLM, specialized in scientific literature understanding. These models demonstrate promising performance on scientific literature understanding benchmarks. Our contributions are threefold: (1) We present an effective framework that integrates CPT and SFT to adapt LLMs to scientific literature understanding, which can also be easily adapted to other domains. (2) We propose an LLM-based synthesis method to generate diverse and high-quality scientific instructions, resulting in a new instruction set -- SciLitIns -- for supervised fine-tuning in less-represented scientific domains. (3) SciLitLLM achieves promising performance improvements on scientific literature understanding benchmarks.","url_abs":"https://arxiv.org/abs/2408.15545v5","url_pdf":"https://arxiv.org/pdf/2408.15545v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scilitllm-how-to-adapt-llms-for-scientific","repo_url":"https://github.com/dptech-corp/Uni-SMART","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"instruction-following","task_name":"Instruction Following"},{"task_slug":"scientific-discovery","task_name":"scientific discovery"}],"methods":[{"method_slug":"set","method_name":"SET"},{"method_slug":"sft","method_name":"SFT"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2408.15545","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.15545"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dptech-corp/Uni-SMART","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":2,"ran":11},"by_repo_kind":{"official":{"samples":13,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"82a2172211c2ce1f","entry":"chunk_text","repo":"dptech-corp/uni-smart","repo_kind":"official","path":"SciLitLLM/cpt/format_grammar_correction.py","file_url":"https://github.com/dptech-corp/uni-smart/blob/HEAD/SciLitLLM/cpt/format_grammar_correction.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"82a2172211c2ce1f"}},{"code_sha256_prefix":"2e366df93e0e0280","entry":"clean_text","repo":"dptech-corp/Uni-SMART","repo_kind":"official","path":"SciLitLLM/sft/helper/parse_pdfs.py","file_url":"https://github.com/dptech-corp/Uni-SMART/blob/HEAD/SciLitLLM/sft/helper/parse_pdfs.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2e366df93e0e0280"}},{"code_sha256_prefix":"4e655fe9976690e7","entry":"get_lev_sim","repo":"dptech-corp/Uni-SMART","repo_kind":"official","path":"SciLitLLM/sft/helper/dedup.py","file_url":"https://github.com/dptech-corp/Uni-SMART/blob/HEAD/SciLitLLM/sft/helper/dedup.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4e655fe9976690e7"}},{"code_sha256_prefix":"6ced88d74c21da6b","entry":"get_qa_pair","repo":"dptech-corp/Uni-SMART","repo_kind":"official","path":"SciLitLLM/sft/helper/infer_sft.py","file_url":"https://github.com/dptech-corp/Uni-SMART/blob/HEAD/SciLitLLM/sft/helper/infer_sft.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6ced88d74c21da6b"}},{"code_sha256_prefix":"c8e533eb7a6418fc","entry":"parse_json_score","repo":"dptech-corp/Uni-SMART","repo_kind":"official","path":"SciLitLLM/sft/helper/infer_sft.py","file_url":"https://github.com/dptech-corp/Uni-SMART/blob/HEAD/SciLitLLM/sft/helper/infer_sft.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c8e533eb7a6418fc"}},{"code_sha256_prefix":"978a0952e0e705b2","entry":"parse_score","repo":"dptech-corp/Uni-SMART","repo_kind":"official","path":"SciLitLLM/cpt/quality_control/llama_infer.py","file_url":"https://github.com/dptech-corp/Uni-SMART/blob/HEAD/SciLitLLM/cpt/quality_control/llama_infer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"978a0952e0e705b2"}},{"code_sha256_prefix":"3b6071d2b3a5c4ef","entry":"process_chunk","repo":"dptech-corp/Uni-SMART","repo_kind":"official","path":"SciLitLLM/cpt/format_grammar_correction.py","file_url":"https://github.com/dptech-corp/Uni-SMART/blob/HEAD/SciLitLLM/cpt/format_grammar_correction.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b6071d2b3a5c4ef"}},{"code_sha256_prefix":"037abb9ab6603b81","entry":"process_text_entry","repo":"dptech-corp/Uni-SMART","repo_kind":"official","path":"SciLitLLM/cpt/format_grammar_correction.py","file_url":"https://github.com/dptech-corp/Uni-SMART/blob/HEAD/SciLitLLM/cpt/format_grammar_correction.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"037abb9ab6603b81"}},{"code_sha256_prefix":"7daf38d1d71fe676","entry":"read_json","repo":"dptech-corp/Uni-SMART","repo_kind":"official","path":"SciLitLLM/sft/helper/dedup.py","file_url":"https://github.com/dptech-corp/Uni-SMART/blob/HEAD/SciLitLLM/sft/helper/dedup.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7daf38d1d71fe676"}},{"code_sha256_prefix":"953b99c2358aec7b","entry":"read_json","repo":"dptech-corp/Uni-SMART","repo_kind":"official","path":"SciLitLLM/sft/helper/infer_sft.py","file_url":"https://github.com/dptech-corp/Uni-SMART/blob/HEAD/SciLitLLM/sft/helper/infer_sft.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"953b99c2358aec7b"}},{"code_sha256_prefix":"793cd3b000794433","entry":"read_jsonl","repo":"dptech-corp/Uni-SMART","repo_kind":"official","path":"SciLitLLM/cpt/quality_control/llama_infer.py","file_url":"https://github.com/dptech-corp/Uni-SMART/blob/HEAD/SciLitLLM/cpt/quality_control/llama_infer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"793cd3b000794433"}},{"code_sha256_prefix":"e73cf9602ff9d98e","entry":"run_batch","repo":"dptech-corp/uni-smart","repo_kind":"official","path":"SciLitLLM/sft/gen_synthetic_data.py","file_url":"https://github.com/dptech-corp/uni-smart/blob/HEAD/SciLitLLM/sft/gen_synthetic_data.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e73cf9602ff9d98e"}},{"code_sha256_prefix":"63649262371c079a","entry":"top_percent_indices","repo":"dptech-corp/Uni-SMART","repo_kind":"official","path":"SciLitLLM/sft/helper/dedup.py","file_url":"https://github.com/dptech-corp/Uni-SMART/blob/HEAD/SciLitLLM/sft/helper/dedup.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"63649262371c079a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}