HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledge

By Haochun Wang, Chi-Liang Liu, Nuwa Xi, Zewen Qiang, Sendong Zhao, Bing Qin, Ting Liu. arXiv.org, 2023

297 citations recorded by the source corpus.

Large Language Models (LLMs), such as the LLaMA model, have demonstrated their effectiveness in various general-domain natural language processing (NLP) tasks. Nevertheless, LLMs have not yet performed optimally in biomedical domain tasks due to the need for medical expertise in the responses. In response to this challenge, we propose HuaTuo, a LLaMA-based model that has been supervised-fine-tuned with generated QA (Question-Answer) instances. The experimental results demonstrate that HuaTuo generates responses that possess more reliable medical knowledge. Our proposed HuaTuo model is accessible at https://github.com/SCIR-HI/Huatuo-Llama-Med-Chinese.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/302ee27524a717ddc21f332ca634b9211c6ec6aa

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jianfeng Gao
2. Chunyuan Li
3. Dacheng Tao
4. Jianwei Yang
5. Haotian Liu
6. Zhe Gan
7. Can Xu
8. Tianyi Zhou

Citations in the corpus

• Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health (2023, background): “In fact, there has already been several studies, such as Clinical Camel [33], DoctorGLM [72], ChatDoctor [34], HuaTuo [73], MedAlpaca [35], that attempt to create clinical chatbots by instruction fine-tuning open source LLMs (e.g., LLaMA) on biomedical corpora.”
• A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics (2023, unknown): “4, 30, 120 Reasoning, Multidisciplinary 11/2022 Org Med-PaLM [99] Flan-PaLM/PaLM 540 CoT, Self-consistency 12/2022 GPT-4-Med [183] GPT-4 no specialized prompt crafting 03/2023 DeID-GPT [184] GPT-4 De-identifying 03/2023 Github ChatDoctor [116] LLaMA 7 Retrieve online, external knowledge 03/2023 Gith”
• HuatuoGPT, towards Taming Language Model to Be a Doctor (2023, methodology): “BenTsao [28]21 is a knowledge-enhanced Chinese Medical LLM trained on over 8K instructions.”
• A Comprehensive Overview of Large Language Models (2023, unknown): “HuaTuo [143] is a medical knowledge model, fine-tuned with a generated QA dataset of 8k instructions.”
• LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day (2023, background): “…by ChatGPT [34]/GPT-4 [35] and the success of open-sourced instruction-tuned large language models (LLMs) in the general domain, several biomedical LLM chatbots have been developed, including ChatDoctor [49], Med-Alpaca [13], PMC-LLaMA [47], Clinical Camel [1], DoctorGLM [48], and Huatuo [46].”
• A Survey on Knowledge Distillation of Large Language Models (2024, background): “Many works have demonstrated that LLMs could be directly used as recommenders without fine-tuning (Wang et al., 2023i; Dai et al., 2023c) or generate auxiliary textual features to benefit recommender systems (Xi et al., 2023; Ren et al., 2023; Wei et al., 2024).”