{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/huatuogpt-ii-one-stage-training-for-medical","title":"HuatuoGPT-II, One-stage Training for Medical Adaption of LLMs","arxiv_id":"2311.09774","date":"2023-11-16","proceeding":null,"authors":["Junying Chen","Xidong Wang","Ke Ji","Anningzhe Gao","Feng Jiang","Shunian Chen","Hongbo Zhang","Dingjie Song","Wenya Xie","Chuyi Kong","Jianquan Li","Xiang Wan","Haizhou Li","Benyou Wang"],"abstract":"Adapting a language model into a specific domain, a.k.a `domain adaption', is a common practice when specialized knowledge, e.g. medicine, is not encapsulated in a general language model like Llama2. The challenge lies in the heterogeneity of data across the two training stages, as it varies in languages, genres, or formats. To tackle this and simplify the learning protocol, we propose to transform heterogeneous data, from the both pre-training and supervised stages, into a unified, simple input-output pair format. We validate the new protocol in the domains where proprietary LLMs like ChatGPT perform relatively poorly, such as Traditional Chinese Medicine. The developed model, HuatuoGPT-II, has shown state-of-the-art performance in Chinese medicine domain on a number of benchmarks, e.g. medical licensing exams. It even outperforms proprietary models like ChatGPT and GPT-4 in some aspects, especially in Traditional Chinese Medicine. Expert manual evaluations further validate HuatuoGPT-II's advantages over existing LLMs. Notably, HuatuoGPT-II was benchmarked in a fresh Chinese National Medical Licensing Examination where it achieved the best performance, showcasing not only its effectiveness but also its generalization capabilities.","url_abs":"https://arxiv.org/abs/2311.09774v2","url_pdf":"https://arxiv.org/pdf/2311.09774v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"huatuogpt-ii-one-stage-training-for-medical","repo_url":"https://github.com/freedomintelligence/huatuogpt-ii","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"domain-adaptation","task_name":"Domain Adaptation"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-4","method_name":"GPT-4"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2311.09774","atlas_url":"https://app.syntology.ai/?focus=2311.09774","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.09774"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/freedomintelligence/huatuogpt-ii","reach":{"status":"ok"}}],"summary":{"ran":11,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":14,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"1c39574f5e470abf","entry":"deduplicate","repo":"freedomintelligence/huatuogpt-ii","repo_kind":"official","path":"evaluation/eval_huatuo_inst.py","file_url":"https://github.com/freedomintelligence/huatuogpt-ii/blob/HEAD/evaluation/eval_huatuo_inst.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1c39574f5e470abf"}},{"code_sha256_prefix":"13ebc20073fe384f","entry":"filter_str","repo":"freedomintelligence/huatuogpt-ii","repo_kind":"official","path":"adaption/data_unification/rewrite.py","file_url":"https://github.com/freedomintelligence/huatuogpt-ii/blob/HEAD/adaption/data_unification/rewrite.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"13ebc20073fe384f"}},{"code_sha256_prefix":"baa2dd07a544df5c","entry":"generate_prompt","repo":"freedomintelligence/huatuogpt-ii","repo_kind":"official","path":"cli_demo.py","file_url":"https://github.com/freedomintelligence/huatuogpt-ii/blob/HEAD/cli_demo.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"baa2dd07a544df5c"}},{"code_sha256_prefix":"dbd2112d465a82e2","entry":"get_data_query","repo":"freedomintelligence/huatuogpt-ii","repo_kind":"official","path":"adaption/data_unification/rewrite.py","file_url":"https://github.com/freedomintelligence/huatuogpt-ii/blob/HEAD/adaption/data_unification/rewrite.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dbd2112d465a82e2"}},{"code_sha256_prefix":"0cc6223401defc70","entry":"get_response","repo":"freedomintelligence/huatuogpt-ii","repo_kind":"official","path":"evaluation/eval_qa.py","file_url":"https://github.com/freedomintelligence/huatuogpt-ii/blob/HEAD/evaluation/eval_qa.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0cc6223401defc70"}},{"code_sha256_prefix":"3da76bec93a6ae0c","entry":"match_choice","repo":"freedomintelligence/huatuogpt-ii","repo_kind":"official","path":"evaluation/scorer.py","file_url":"https://github.com/freedomintelligence/huatuogpt-ii/blob/HEAD/evaluation/scorer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3da76bec93a6ae0c"}},{"code_sha256_prefix":"3e574889ab383717","entry":"match_choice2","repo":"freedomintelligence/huatuogpt-ii","repo_kind":"official","path":"evaluation/scorer.py","file_url":"https://github.com/freedomintelligence/huatuogpt-ii/blob/HEAD/evaluation/scorer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3e574889ab383717"}},{"code_sha256_prefix":"c0faf671597614bb","entry":"match_choice3","repo":"freedomintelligence/huatuogpt-ii","repo_kind":"official","path":"evaluation/scorer.py","file_url":"https://github.com/freedomintelligence/huatuogpt-ii/blob/HEAD/evaluation/scorer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c0faf671597614bb"}},{"code_sha256_prefix":"2af3169badbc6691","entry":"ngram_jaccard_score","repo":"freedomintelligence/huatuogpt-ii","repo_kind":"official","path":"adaption/data_unification/rewrite.py","file_url":"https://github.com/freedomintelligence/huatuogpt-ii/blob/HEAD/adaption/data_unification/rewrite.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2af3169badbc6691"}},{"code_sha256_prefix":"afe2c8247569f6e0","entry":"parse_classification_cot","repo":"freedomintelligence/huatuogpt-ii","repo_kind":"official","path":"evaluation/eval_huatuo_conv.py","file_url":"https://github.com/freedomintelligence/huatuogpt-ii/blob/HEAD/evaluation/eval_huatuo_conv.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"afe2c8247569f6e0"}},{"code_sha256_prefix":"3bc9437adf1d7ea1","entry":"remove_overlap","repo":"freedomintelligence/huatuogpt-ii","repo_kind":"official","path":"cli_demo.py","file_url":"https://github.com/freedomintelligence/huatuogpt-ii/blob/HEAD/cli_demo.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3bc9437adf1d7ea1"}},{"code_sha256_prefix":"4b9933d832571ead","entry":"table_to_csv_string","repo":"freedomintelligence/huatuogpt-ii","repo_kind":"official","path":"evaluation/eval_qa.py","file_url":"https://github.com/freedomintelligence/huatuogpt-ii/blob/HEAD/evaluation/eval_qa.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4b9933d832571ead"}},{"code_sha256_prefix":"e420c2ab67f141f8","entry":"transfer_conv","repo":"freedomintelligence/huatuogpt-ii","repo_kind":"official","path":"evaluation/eval_huatuo_conv.py","file_url":"https://github.com/freedomintelligence/huatuogpt-ii/blob/HEAD/evaluation/eval_huatuo_conv.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e420c2ab67f141f8"}},{"code_sha256_prefix":"719c8f0c667de29a","entry":"load_model","repo":"freedomintelligence/huatuogpt-ii","repo_kind":"official","path":"cli_demo.py","file_url":"https://github.com/freedomintelligence/huatuogpt-ii/blob/HEAD/cli_demo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"719c8f0c667de29a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}