DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Task

• URL: http://arxiv.org/abs/2304.01097v2
• Date: Mon, 17 Apr 2023 17:06:29 GMT
• Title: DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Task
• Authors: Honglin Xiong, Sheng Wang, Yitao Zhu, Zihao Zhao, Yuxiao Liu, Linlin Huang, Qian Wang, Dinggang Shen
• Abstract summary: Large language models (LLMs) typically perform better in English and have not been explicitly trained for the medical domain. We have collected databases of medical dialogues in Chinese with ChatGPT's help and adopted several techniques to train an easy-deploy LLM. DoctorGLM is currently an early-stage engineering attempt and contain various mistakes.
• Score: 44.21600465230548
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The recent progress of large language models (LLMs), including ChatGPT and GPT-4, in comprehending and responding to human instructions has been remarkable. Nevertheless, these models typically perform better in English and have not been explicitly trained for the medical domain, resulting in suboptimal precision in diagnoses, drug recommendations, and other medical advice. Additionally, training and deploying a dialogue model is still believed to be impossible for hospitals, hindering the promotion of LLMs. To tackle these challenges, we have collected databases of medical dialogues in Chinese with ChatGPT's help and adopted several techniques to train an easy-deploy LLM. Remarkably, we were able to fine-tune the ChatGLM-6B on a single A100 80G in 13 hours, which means having a healthcare-purpose LLM can be very affordable. DoctorGLM is currently an early-stage engineering attempt and contain various mistakes. We are sharing it with the broader community to invite feedback and suggestions to improve its healthcare-focused capabilities: https://github.com/xionghonglin/DoctorGLM.

Related papers

• MedG-KRP: Medical Graph Knowledge Representation Probing [0.6496030410305753]
  Large language models (LLMs) have recently emerged as powerful tools, finding many medical applications. We introduce a knowledge graph (KG)-based method to evaluate the biomedical reasoning abilities of LLMs. We test GPT-4, Llama3-70b, and PalmyraMed-70b, a specialized medical model.
  arXiv  Detail & Related papers  (2024-12-14T22:23:20Z)
• Demystifying Large Language Models for Medicine: A Primer [50.83806796466396]
  Large language models (LLMs) represent a transformative class of AI tools capable of revolutionizing various aspects of healthcare. This tutorial aims to equip healthcare professionals with the tools necessary to effectively integrate LLMs into clinical practice.
  arXiv  Detail & Related papers  (2024-10-24T15:41:56Z)
• LLMs for Doctors: Leveraging Medical LLMs to Assist Doctors, Not Replace Them [41.65016162783525]
  We focus on tuning the Large Language Models to be medical assistants who collaborate with more experienced doctors. We construct a Chinese medical dataset called DoctorFLAN to support the entire workflow of doctors. We evaluate LLMs in doctor-oriented scenarios by constructing the DoctorFLAN-textittest containing 550 single-turn Q&A and DotaBench containing 74 multi-turn conversations.
  arXiv  Detail & Related papers  (2024-06-26T03:08:24Z)
• Large Language Model Distilling Medication Recommendation Model [58.94186280631342]
  We harness the powerful semantic comprehension and input-agnostic characteristics of Large Language Models (LLMs) Our research aims to transform existing medication recommendation methodologies using LLMs. To mitigate this, we have developed a feature-level knowledge distillation technique, which transfers the LLM's proficiency to a more compact model.
  arXiv  Detail & Related papers  (2024-02-05T08:25:22Z)
• ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences [51.66185471742271]
  We propose ChiMed-GPT, a benchmark LLM designed explicitly for Chinese medical domain. ChiMed-GPT undergoes a comprehensive training regime with pre-training, SFT, and RLHF. We analyze possible biases through prompting ChiMed-GPT to perform attitude scales regarding discrimination of patients.
  arXiv  Detail & Related papers  (2023-11-10T12:25:32Z)
• A Survey of Large Language Models in Medicine: Progress, Application, and Challenge [85.09998659355038]
  Large language models (LLMs) have received substantial attention due to their capabilities for understanding and generating human language. This review aims to provide a detailed overview of the development and deployment of LLMs in medicine.
  arXiv  Detail & Related papers  (2023-11-09T02:55:58Z)
• MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records [60.35217378132709]
  Large language models (LLMs) can follow natural language instructions with human-level fluency. evaluating LLMs on realistic text generation tasks for healthcare remains challenging. We introduce MedAlign, a benchmark dataset of 983 natural language instructions for EHR data.
  arXiv  Detail & Related papers  (2023-08-27T12:24:39Z)
• IvyGPT: InteractiVe Chinese pathwaY language model in medical domain [7.5386393444603454]
  General large language models (LLMs) such as ChatGPT have shown remarkable success. We propose IvyGPT, an LLM based on LLaMA that is trained and fine-tuned with high-quality medical question-answer. In the training, we used QLoRA to train 33 billion parameters on a small number of NVIDIA A100 (80GB) Experimental results show that IvyGPT has outperformed other medical GPT models.
  arXiv  Detail & Related papers  (2023-07-20T01:11:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.