Integrating Physician Diagnostic Logic into Large Language Models:
Preference Learning from Process Feedback
- URL: http://arxiv.org/abs/2401.05695v1
- Date: Thu, 11 Jan 2024 06:42:45 GMT
- Title: Integrating Physician Diagnostic Logic into Large Language Models:
Preference Learning from Process Feedback
- Authors: Chengfeng Dou, Zhi Jin, Wenpin Jiao, Haiyan Zhao, Yongqiang Zhao,
Zhenwei Tao
- Abstract summary: We propose an approach called preference learning from process feedback.
PLPF integrates the doctor's diagnostic logic into LLMs.
We show that PLPF enhances the diagnostic accuracy of the baseline model in medical conversations by 17.6%.
- Score: 20.73076574974894
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The use of large language models in medical dialogue generation has garnered
significant attention, with a focus on improving response quality and fluency.
While previous studies have made progress in optimizing model performance for
single-round medical Q&A tasks, there is a need to enhance the model's
capability for multi-round conversations to avoid logical inconsistencies. To
address this, we propose an approach called preference learning from process
feedback~(PLPF), which integrates the doctor's diagnostic logic into LLMs. PLPF
involves rule modeling, preference data generation, and preference alignment to
train the model to adhere to the diagnostic process. Experimental results using
Standardized Patient Testing show that PLPF enhances the diagnostic accuracy of
the baseline model in medical conversations by 17.6%, outperforming traditional
reinforcement learning from human feedback. Additionally, PLPF demonstrates
effectiveness in both multi-round and single-round dialogue tasks, showcasing
its potential for improving medical dialogue generation.
Related papers
- Dr-LLaVA: Visual Instruction Tuning with Symbolic Clinical Grounding [53.629132242389716]
Vision-Language Models (VLM) can support clinicians by analyzing medical images and engaging in natural language interactions.
VLMs often exhibit "hallucinogenic" behavior, generating textual outputs not grounded in contextual multimodal information.
We propose a new alignment algorithm that uses symbolic representations of clinical reasoning to ground VLMs in medical knowledge.
arXiv Detail & Related papers (2024-05-29T23:19:28Z) - MRC-based Nested Medical NER with Co-prediction and Adaptive Pre-training [0.38498367961730184]
We propose a medical NER model based on Machine Reading (MRC), which uses a task-adaptive pre-training strategy to improve the model's capability in the medical field.
Our proposed model outperforms the compared state-of-the-art (SOTA) models.
arXiv Detail & Related papers (2024-03-23T11:14:02Z) - Customizing General-Purpose Foundation Models for Medical Report
Generation [64.31265734687182]
The scarcity of labelled medical image-report pairs presents great challenges in the development of deep and large-scale neural networks.
We propose customizing off-the-shelf general-purpose large-scale pre-trained models, i.e., foundation models (FMs) in computer vision and natural language processing.
arXiv Detail & Related papers (2023-06-09T03:02:36Z) - PlugMed: Improving Specificity in Patient-Centered Medical Dialogue
Generation using In-Context Learning [20.437165038293426]
The patient-centered medical dialogue systems strive to offer diagnostic interpretation services to users who are less knowledgeable about medical knowledge.
It is difficult for the large language models (LLMs) to guarantee the specificity of responses in spite of its promising performance.
Inspired by in-context learning, we propose PlugMed, a Plug-and-Play Medical Dialogue System.
arXiv Detail & Related papers (2023-05-19T08:18:24Z) - Large Language Models for Healthcare Data Augmentation: An Example on
Patient-Trial Matching [49.78442796596806]
We propose an innovative privacy-aware data augmentation approach for patient-trial matching (LLM-PTM)
Our experiments demonstrate a 7.32% average improvement in performance using the proposed LLM-PTM method, and the generalizability to new data is improved by 12.12%.
arXiv Detail & Related papers (2023-03-24T03:14:00Z) - DR.BENCH: Diagnostic Reasoning Benchmark for Clinical Natural Language
Processing [5.022185333260402]
Diagnostic Reasoning Benchmarks, DR.BENCH, is a new benchmark for developing and evaluating cNLP models with clinical diagnostic reasoning ability.
DR.BENCH is the first clinical suite of tasks designed to be a natural language generation framework to evaluate pre-trained language models.
arXiv Detail & Related papers (2022-09-29T16:05:53Z) - An Evaluation of Generative Pre-Training Model-based Therapy Chatbot for
Caregivers [5.2116528363639985]
Generative-based approaches, such as the OpenAI GPT models, could allow for more dynamic conversations in therapy contexts.
We built a chatbots using the GPT-2 model and fine-tuned it with 306 therapy session transcripts between family caregivers of individuals with dementia and therapists conducting Problem Solving Therapy.
Results showed that the fine-tuned model created more non-word outputs than the pre-trained model.
arXiv Detail & Related papers (2021-07-28T01:01:08Z) - Learning ULMFiT and Self-Distillation with Calibration for Medical
Dialogue System [2.055949720959582]
In recent years, the introduction of state-of-the-art deep learning models and transfer learning techniques have contributed to the performance of NLP tasks.
Some deep neural networks are poorly calibrated and wrongly estimate the uncertainty.
In this paper, we investigate the well-calibrated model for ULMFiT and self-distillation in a medical dialogue system.
arXiv Detail & Related papers (2021-07-20T17:11:24Z) - Semi-Supervised Variational Reasoning for Medical Dialogue Generation [70.838542865384]
Two key characteristics are relevant for medical dialogue generation: patient states and physician actions.
We propose an end-to-end variational reasoning approach to medical dialogue generation.
A physician policy network composed of an action-classifier and two reasoning detectors is proposed for augmented reasoning ability.
arXiv Detail & Related papers (2021-05-13T04:14:35Z) - Adversarial Sample Enhanced Domain Adaptation: A Case Study on
Predictive Modeling with Electronic Health Records [57.75125067744978]
We propose a data augmentation method to facilitate domain adaptation.
adversarially generated samples are used during domain adaptation.
Results confirm the effectiveness of our method and the generality on different tasks.
arXiv Detail & Related papers (2021-01-13T03:20:20Z) - MedDG: An Entity-Centric Medical Consultation Dataset for Entity-Aware
Medical Dialogue Generation [86.38736781043109]
We build and release a large-scale high-quality Medical Dialogue dataset related to 12 types of common Gastrointestinal diseases named MedDG.
We propose two kinds of medical dialogue tasks based on MedDG dataset. One is the next entity prediction and the other is the doctor response generation.
Experimental results show that the pre-train language models and other baselines struggle on both tasks with poor performance in our dataset.
arXiv Detail & Related papers (2020-10-15T03:34:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.