Generating medically-accurate summaries of patient-provider dialogue: A multi-stage approach using large language models

• URL: http://arxiv.org/abs/2305.05982v1
• Date: Wed, 10 May 2023 08:48:53 GMT
• Title: Generating medically-accurate summaries of patient-provider dialogue: A multi-stage approach using large language models
• Authors: Varun Nair, Elliot Schumacher, Anitha Kannan
• Abstract summary: An effective summary is required to be coherent and accurately capture all the medically relevant information in the dialogue. This paper tackles the problem of medical conversation summarization by discretizing the task into several smaller dialogue-understanding tasks.
• Score: 6.252236971703546
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A medical provider's summary of a patient visit serves several critical purposes, including clinical decision-making, facilitating hand-offs between providers, and as a reference for the patient. An effective summary is required to be coherent and accurately capture all the medically relevant information in the dialogue, despite the complexity of patient-generated language. Even minor inaccuracies in visit summaries (for example, summarizing "patient does not have a fever" when a fever is present) can be detrimental to the outcome of care for the patient. This paper tackles the problem of medical conversation summarization by discretizing the task into several smaller dialogue-understanding tasks that are sequentially built upon. First, we identify medical entities and their affirmations within the conversation to serve as building blocks. We study dynamically constructing few-shot prompts for tasks by conditioning on relevant patient information and use GPT-3 as the backbone for our experiments. We also develop GPT-derived summarization metrics to measure performance against reference summaries quantitatively. Both our human evaluation study and metrics for medical correctness show that summaries generated using this approach are clinically accurate and outperform the baseline approach of summarizing the dialog in a zero-shot, single-prompt setting.

Related papers

• Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarization [8.456700096020601]
  Large language models (LLMs) have shown promise in natural language processing (NLP), but their effectiveness on a diverse range of clinical summarization tasks remains unproven. In this study, we apply adaptation methods to eight LLMs, spanning four distinct clinical summarization tasks. A clinical reader study with ten physicians evaluates summary, completeness, correctness, and conciseness; in a majority of cases, summaries from our best adapted LLMs are either equivalent (45%) or superior (36%) compared to summaries from medical experts.
  arXiv  Detail & Related papers  (2023-09-14T05:15:01Z)
• MedNgage: A Dataset for Understanding Engagement in Patient-Nurse Conversations [4.847266237348932]
  Patients who effectively manage their symptoms often demonstrate higher levels of engagement in conversations and interventions with healthcare practitioners. It is crucial for AI systems to understand the engagement in natural conversations between patients and practitioners to better contribute toward patient care. We present a novel dataset (MedNgage) which consists of patient-nurse conversations about cancer symptom management.
  arXiv  Detail & Related papers  (2023-05-31T16:06:07Z)
• SPeC: A Soft Prompt-Based Calibration on Performance Variability of Large Language Model in Clinical Notes Summarization [50.01382938451978]
  We introduce a model-agnostic pipeline that employs soft prompts to diminish variance while preserving the advantages of prompt-based summarization. Experimental findings indicate that our method not only bolsters performance but also effectively curbs variance for various language models.
  arXiv  Detail & Related papers  (2023-03-23T04:47:46Z)
• A Meta-Evaluation of Faithfulness Metrics for Long-Form Hospital-Course Summarization [2.8575516056239576]
  Long-form clinical summarization of hospital admissions has real-world significance because of its potential to help both clinicians and patients. We benchmark faithfulness metrics against fine-grained human annotations for model-generated summaries of a patient's Brief Hospital Course.
  arXiv  Detail & Related papers  (2023-03-07T14:57:06Z)
• Discharge Summary Hospital Course Summarisation of In Patient Electronic Health Record Text with Clinical Concept Guided Deep Pre-Trained Transformer Models [1.1393603788068778]
  Brief Hospital Course (BHC) summaries are succinct summaries of an entire hospital encounter, embedded within discharge summaries. We demonstrate a range of methods for BHC summarisation demonstrating the performance of deep learning summarisation models.
  arXiv  Detail & Related papers  (2022-11-14T05:39:45Z)
• Towards more patient friendly clinical notes through language models and ontologies [57.51898902864543]
  We present a novel approach to automated medical text based on word simplification and language modelling. We use a new dataset pairs of publicly available medical sentences and a version of them simplified by clinicians. Our method based on a language model trained on medical forum data generates simpler sentences while preserving both grammar and the original meaning.
  arXiv  Detail & Related papers  (2021-12-23T16:11:19Z)
• Self-supervised Answer Retrieval on Clinical Notes [68.87777592015402]
  We introduce CAPR, a rule-based self-supervision objective for training Transformer language models for domain-specific passage matching. We apply our objective in four Transformer-based architectures: Contextual Document Vectors, Bi-, Poly- and Cross-encoders. We report that CAPR outperforms strong baselines in the retrieval of domain-specific passages and effectively generalizes across rule-based and human-labeled passages.
  arXiv  Detail & Related papers  (2021-08-02T10:42:52Z)
• Semi-Supervised Variational Reasoning for Medical Dialogue Generation [70.838542865384]
  Two key characteristics are relevant for medical dialogue generation: patient states and physician actions. We propose an end-to-end variational reasoning approach to medical dialogue generation. A physician policy network composed of an action-classifier and two reasoning detectors is proposed for augmented reasoning ability.
  arXiv  Detail & Related papers  (2021-05-13T04:14:35Z)
• Benchmarking Automated Clinical Language Simplification: Dataset, Algorithm, and Evaluation [48.87254340298189]
  We construct a new dataset named MedLane to support the development and evaluation of automated clinical language simplification approaches. We propose a new model called DECLARE that follows the human annotation procedure and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-12-04T06:09:02Z)
• MedDG: An Entity-Centric Medical Consultation Dataset for Entity-Aware Medical Dialogue Generation [86.38736781043109]
  We build and release a large-scale high-quality Medical Dialogue dataset related to 12 types of common Gastrointestinal diseases named MedDG. We propose two kinds of medical dialogue tasks based on MedDG dataset. One is the next entity prediction and the other is the doctor response generation. Experimental results show that the pre-train language models and other baselines struggle on both tasks with poor performance in our dataset.
  arXiv  Detail & Related papers  (2020-10-15T03:34:33Z)
• Dr. Summarize: Global Summarization of Medical Dialogue by Exploiting Local Structures [4.3012765978447565]
  We present a novel approach to medical conversation summarization that leverages the unique and independent local structures created when gathering a patient's medical history. Our approach is preferred on twice the number of pointers to the baseline pointer generator model and captures most or all of the information in 80% of the conversations.
  arXiv  Detail & Related papers  (2020-09-18T07:35:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.