IvyGPT: InteractiVe Chinese pathwaY language model in medical domain

• URL: http://arxiv.org/abs/2307.10512v1
• Date: Thu, 20 Jul 2023 01:11:14 GMT
• Title: IvyGPT: InteractiVe Chinese pathwaY language model in medical domain
• Authors: Rongsheng Wang and Yaofei Duan and ChanTong Lam and Jiexi Chen and Jiangsheng Xu and Haoming Chen and Xiaohong Liu and Patrick Cheong-Iao Pang and Tao Tan
• Abstract summary: General large language models (LLMs) such as ChatGPT have shown remarkable success. We propose IvyGPT, an LLM based on LLaMA that is trained and fine-tuned with high-quality medical question-answer. In the training, we used QLoRA to train 33 billion parameters on a small number of NVIDIA A100 (80GB) Experimental results show that IvyGPT has outperformed other medical GPT models.
• Score: 7.5386393444603454
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: General large language models (LLMs) such as ChatGPT have shown remarkable success. However, such LLMs have not been widely adopted for medical purposes, due to poor accuracy and inability to provide medical advice. We propose IvyGPT, an LLM based on LLaMA that is trained and fine-tuned with high-quality medical question-answer (QA) instances and Reinforcement Learning from Human Feedback (RLHF). After supervised fine-tuning, IvyGPT has good multi-turn conversation capabilities, but it cannot perform like a doctor in other aspects, such as comprehensive diagnosis. Through RLHF, IvyGPT can output richer diagnosis and treatment answers that are closer to human. In the training, we used QLoRA to train 33 billion parameters on a small number of NVIDIA A100 (80GB) GPUs. Experimental results show that IvyGPT has outperformed other medical GPT models.

Related papers

• MedG-KRP: Medical Graph Knowledge Representation Probing [0.6496030410305753]
  Large language models (LLMs) have recently emerged as powerful tools, finding many medical applications. We introduce a knowledge graph (KG)-based method to evaluate the biomedical reasoning abilities of LLMs. We test GPT-4, Llama3-70b, and PalmyraMed-70b, a specialized medical model.
  arXiv  Detail & Related papers  (2024-12-14T22:23:20Z)
• MedExpQA: Multilingual Benchmarking of Large Language Models for Medical Question Answering [8.110978727364397]
  Large Language Models (LLMs) have the potential of facilitating the development of Artificial Intelligence technology. This paper presents MedExpQA, the first multilingual benchmark based on medical exams to evaluate LLMs in Medical Question Answering.
  arXiv  Detail & Related papers  (2024-04-08T15:03:57Z)
• BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Text [82.7001841679981]
  BioMedLM is a 2.7 billion parameter GPT-style autoregressive model trained exclusively on PubMed abstracts and full articles. When fine-tuned, BioMedLM can produce strong multiple-choice biomedical question-answering results competitive with larger models. BioMedLM can also be fine-tuned to produce useful answers to patient questions on medical topics.
  arXiv  Detail & Related papers  (2024-03-27T10:18:21Z)
• ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences [51.66185471742271]
  We propose ChiMed-GPT, a benchmark LLM designed explicitly for Chinese medical domain. ChiMed-GPT undergoes a comprehensive training regime with pre-training, SFT, and RLHF. We analyze possible biases through prompting ChiMed-GPT to perform attitude scales regarding discrimination of patients.
  arXiv  Detail & Related papers  (2023-11-10T12:25:32Z)
• Augmenting Black-box LLMs with Medical Textbooks for Biomedical Question Answering (Published in Findings of EMNLP 2024) [48.17095875619711]
  We present a system called LLMs Augmented with Medical Textbooks (LLM-AMT) LLM-AMT integrates authoritative medical textbooks into the LLMs' framework using plug-and-play modules. We found that medical textbooks as a retrieval corpus is proven to be a more effective knowledge database than Wikipedia in the medical domain.
  arXiv  Detail & Related papers  (2023-09-05T13:39:38Z)
• HuatuoGPT, towards Taming Language Model to Be a Doctor [67.96794664218318]
  HuatuoGPT is a large language model (LLM) for medical consultation. We leverage both textitdistilled data from ChatGPT and textitreal-world data from doctors in the supervised fine-tuned stage.
  arXiv  Detail & Related papers  (2023-05-24T11:56:01Z)
• Large Language Models Leverage External Knowledge to Extend Clinical Insight Beyond Language Boundaries [48.48630043740588]
  Large Language Models (LLMs) such as ChatGPT and Med-PaLM have excelled in various medical question-answering tasks. We develop a novel in-context learning framework to enhance their performance.
  arXiv  Detail & Related papers  (2023-05-17T12:31:26Z)
• DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Task [44.21600465230548]
  Large language models (LLMs) typically perform better in English and have not been explicitly trained for the medical domain. We have collected databases of medical dialogues in Chinese with ChatGPT's help and adopted several techniques to train an easy-deploy LLM. DoctorGLM is currently an early-stage engineering attempt and contain various mistakes.
  arXiv  Detail & Related papers  (2023-04-03T15:57:51Z)
• Evaluating Large Language Models on a Highly-specialized Topic, Radiation Oncology Physics [9.167699167689369]
  This paper proposes evaluating LLMs on a highly-specialized topic, radiation oncology physics. We developed an exam consisting of 100 radiation oncology physics questions. ChatGPT (GPT-3.5), ChatGPT (GPT-4), Bard (LaMDA), and BLOOMZ, were evaluated against medical physicists and non-experts.
  arXiv  Detail & Related papers  (2023-04-01T06:04:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.