Hengqin-RA-v1: Advanced Large Language Model for Diagnosis and Treatment of Rheumatoid Arthritis with Dataset based Traditional Chinese Medicine

• URL: http://arxiv.org/abs/2501.02471v1
• Date: Sun, 05 Jan 2025 07:46:51 GMT
• Title: Hengqin-RA-v1: Advanced Large Language Model for Diagnosis and Treatment of Rheumatoid Arthritis with Dataset based Traditional Chinese Medicine
• Authors: Yishen Liu, Shengda Luo, Zishao Zhong, Tongtong Wu, Jianguo Zhang, Peiyao Ou, Yong Liang, Liang Liu, Hudan Pan,
• Abstract summary: This paper introduces Hengqin-RA-v1, the first large language model specifically tailored for Traditional Chinese Medicine (TCM) We also present HQ-GCM-RA-C1, a comprehensive RA-specific dataset curated from ancient Chinese medical literature, classical texts, and modern clinical studies.
• Score: 9.423846262482716
• License:
• Abstract: Large language models (LLMs) primarily trained on English texts, often face biases and inaccuracies in Chinese contexts. Their limitations are pronounced in fields like Traditional Chinese Medicine (TCM), where cultural and clinical subtleties are vital, further hindered by a lack of domain-specific data, such as rheumatoid arthritis (RA). To address these issues, this paper introduces Hengqin-RA-v1, the first large language model specifically tailored for TCM with a focus on diagnosing and treating RA. We also present HQ-GCM-RA-C1, a comprehensive RA-specific dataset curated from ancient Chinese medical literature, classical texts, and modern clinical studies. This dataset empowers Hengqin-RA-v1 to deliver accurate and culturally informed responses, effectively bridging the gaps left by general-purpose models. Extensive experiments demonstrate that Hengqin-RA-v1 outperforms state-of-the-art models, even surpassing the diagnostic accuracy of TCM practitioners in certain cases.

Related papers

• Technical Report: Small Language Model for Japanese Clinical and Medicine [0.0]
  This report presents a small language model (SLM) for Japanese clinical and medicine, named NCVC-slm-1. In comparison to other large language models, a fine-tuning NCVC-slm-1 demonstrated the highest scores on 6 tasks of total 8 on JMED-LLM.
  arXiv  Detail & Related papers  (2024-12-21T01:12:48Z)
• HC-LLM: Historical-Constrained Large Language Models for Radiology Report Generation [89.3260120072177]
  We propose a novel Historical-Constrained Large Language Models (HC-LLM) framework for Radiology report generation. Our approach extracts both time-shared and time-specific features from longitudinal chest X-rays and diagnostic reports to capture disease progression. Notably, our approach performs well even without historical data during testing and can be easily adapted to other multimodal large models.
  arXiv  Detail & Related papers  (2024-12-15T06:04:16Z)
• BianCang: A Traditional Chinese Medicine Large Language Model [22.582027277167047]
  BianCang is a TCM-specific large language model (LLMs) that first injects domain-specific knowledge and then aligns it through targeted stimulation. We constructed pre-training corpora, instruction-aligned datasets based on real hospital records, and the ChP-TCM dataset derived from the Pharmacopoeia of the People's Republic of China. We compiled extensive TCM and medical corpora for continuous pre-training and supervised fine-tuning, building a comprehensive dataset to refine the model's understanding of TCM.
  arXiv  Detail & Related papers  (2024-11-17T10:17:01Z)
• Comprehensive Study on German Language Models for Clinical and Biomedical Text Understanding [16.220303664681172]
  We pre-trained several German medical language models on 2.4B tokens derived from translated public English medical data and 3B tokens of German clinical data. The resulting models were evaluated on various German downstream tasks, including named entity recognition (NER), multi-label classification, and extractive question answering. We conclude that continuous pre-training has demonstrated the ability to match or even exceed the performance of clinical models trained from scratch.
  arXiv  Detail & Related papers  (2024-04-08T17:24:04Z)
• HuatuoGPT-II, One-stage Training for Medical Adaption of LLMs [61.41790586411816]
  HuatuoGPT-II has shown state-of-the-art performance in Chinese medicine domain on a number of benchmarks. It even outperforms proprietary models like ChatGPT and GPT-4 in some aspects, especially in Traditional Chinese Medicine.
  arXiv  Detail & Related papers  (2023-11-16T10:56:24Z)
• TCM-GPT: Efficient Pre-training of Large Language Models for Domain Adaptation in Traditional Chinese Medicine [11.537289359051975]
  We propose a novel TCMDA (TCM Domain Adaptation) approach, efficient pre-training with domain-specific corpus. Specifically, we first construct a large TCM-specific corpus, TCM-Corpus-1B, by identifying domain keywords and retreving from general corpus. Then, our TCMDA leverages the LoRA which freezes the pretrained model's weights and uses rank decomposition matrices to efficiently train specific dense layers for pre-training and fine-tuning.
  arXiv  Detail & Related papers  (2023-11-03T08:54:50Z)
• PMC-LLaMA: Towards Building Open-source Language Models for Medicine [62.39105735933138]
  Large Language Models (LLMs) have showcased remarkable capabilities in natural language understanding. LLMs struggle in domains that require precision, such as medical applications, due to their lack of domain-specific knowledge. We describe the procedure for building a powerful, open-source language model specifically designed for medicine applications, termed as PMC-LLaMA.
  arXiv  Detail & Related papers  (2023-04-27T18:29:05Z)
• TCM-SD: A Benchmark for Probing Syndrome Differentiation via Natural Language Processing [31.190757020836656]
  We focus on the core task of the TCM diagnosis and treatment system -- syndrome differentiation (SD) Our dataset contains 54,152 real-world clinical records covering 148 syndromes. We propose a domain-specific pre-trained language model, called ZY-BERT.
  arXiv  Detail & Related papers  (2022-03-21T09:59:54Z)
• Medical-VLBERT: Medical Visual Language BERT for COVID-19 CT Report Generation With Alternate Learning [70.71564065885542]
  We propose to use the medical visual language BERT (Medical-VLBERT) model to identify the abnormality on the COVID-19 scans. This model adopts an alternate learning strategy with two procedures that are knowledge pretraining and transferring. For automatic medical report generation on the COVID-19 cases, we constructed a dataset of 368 medical findings in Chinese and 1104 chest CT scans.
  arXiv  Detail & Related papers  (2021-08-11T07:12:57Z)
• CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark [51.38557174322772]
  We present the first Chinese Biomedical Language Understanding Evaluation benchmark. It is a collection of natural language understanding tasks including named entity recognition, information extraction, clinical diagnosis normalization, single-sentence/sentence-pair classification. We report empirical results with the current 11 pre-trained Chinese models, and experimental results show that state-of-the-art neural models perform by far worse than the human ceiling.
  arXiv  Detail & Related papers  (2021-06-15T12:25:30Z)
• Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-based Neural Networks [62.9447303059342]
  We show the importance of this problem in medical community. We present a modification of Bidirectional Representations from Transformers (BERT) model for classification sequence. We use a large-scale Russian EHR dataset consisting of about 4 million unique patient visits.
  arXiv  Detail & Related papers  (2020-07-15T09:22:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.