Enhancing Summarization Performance through Transformer-Based Prompt Engineering in Automated Medical Reporting

• URL: http://arxiv.org/abs/2311.13274v2
• Date: Fri, 19 Jan 2024 10:06:50 GMT
• Title: Enhancing Summarization Performance through Transformer-Based Prompt Engineering in Automated Medical Reporting
• Authors: Daphne van Zandvoort, Laura Wiersema, Tom Huibers, Sandra van Dulmen, Sjaak Brinkkemper
• Abstract summary: Two-shot prompting approach in combination with scope and domain context outperforms other methods. The automated reports are approximately twice as long as the human references.
• Score: 0.49478969093606673
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Customized medical prompts enable Large Language Models (LLM) to effectively address medical dialogue summarization. The process of medical reporting is often time-consuming for healthcare professionals. Implementing medical dialogue summarization techniques presents a viable solution to alleviate this time constraint by generating automated medical reports. The effectiveness of LLMs in this process is significantly influenced by the formulation of the prompt, which plays a crucial role in determining the quality and relevance of the generated reports. In this research, we used a combination of two distinct prompting strategies, known as shot prompting and pattern prompting to enhance the performance of automated medical reporting. The evaluation of the automated medical reports is carried out using the ROUGE score and a human evaluation with the help of an expert panel. The two-shot prompting approach in combination with scope and domain context outperforms other methods and achieves the highest score when compared to the human reference set by a general practitioner. However, the automated reports are approximately twice as long as the human references, due to the addition of both redundant and relevant statements that are added to the report.

Related papers

• MedInsight: A Multi-Source Context Augmentation Framework for Generating Patient-Centric Medical Responses using Large Language Models [3.0874677990361246]
  Large Language Models (LLMs) have shown impressive capabilities in generating human-like responses. We propose MedInsight:a novel retrieval framework that augments LLM inputs with relevant background information. Experiments on the MTSamples dataset validate MedInsight's effectiveness in generating contextually appropriate medical responses.
  arXiv  Detail & Related papers  (2024-03-13T15:20:30Z)
• MKA: A Scalable Medical Knowledge Assisted Mechanism for Generative Models on Medical Conversation Tasks [3.9571320117430866]
  The mechanism aims to assist general neural generative models to achieve better performance on the medical conversation task. The medical-specific knowledge graph is designed within the mechanism, which contains 6 types of medical-related information. The evaluation results demonstrate that models combined with our mechanism outperform original methods in multiple automatic evaluation metrics.
  arXiv  Detail & Related papers  (2023-12-05T04:55:54Z)
• Comparative Experimentation of Accuracy Metrics in Automated Medical Reporting: The Case of Otitis Consultations [0.5242869847419834]
  Generative Artificial Intelligence can be used to automatically generate medical reports based on transcripts of medical consultations. The accuracy of the generated reports needs to be established to ensure their correctness and usefulness. There are several metrics for measuring the accuracy of AI generated reports, but little work has been done towards the application of these metrics in medical reporting.
  arXiv  Detail & Related papers  (2023-11-22T09:51:43Z)
• Generating medically-accurate summaries of patient-provider dialogue: A multi-stage approach using large language models [6.252236971703546]
  An effective summary is required to be coherent and accurately capture all the medically relevant information in the dialogue. This paper tackles the problem of medical conversation summarization by discretizing the task into several smaller dialogue-understanding tasks.
  arXiv  Detail & Related papers  (2023-05-10T08:48:53Z)
• SPeC: A Soft Prompt-Based Calibration on Performance Variability of Large Language Model in Clinical Notes Summarization [50.01382938451978]
  We introduce a model-agnostic pipeline that employs soft prompts to diminish variance while preserving the advantages of prompt-based summarization. Experimental findings indicate that our method not only bolsters performance but also effectively curbs variance for various language models.
  arXiv  Detail & Related papers  (2023-03-23T04:47:46Z)
• A Benchmark for Automatic Medical Consultation System: Frameworks, Tasks and Datasets [70.32630628211803]
  We propose two frameworks to support automatic medical consultation, namely doctor-patient dialogue understanding and task-oriented interaction. A new large medical dialogue dataset with multi-level fine-grained annotations is introduced. We report a set of benchmark results for each task, which shows the usability of the dataset and sets a baseline for future studies.
  arXiv  Detail & Related papers  (2022-04-19T16:43:21Z)
• Towards more patient friendly clinical notes through language models and ontologies [57.51898902864543]
  We present a novel approach to automated medical text based on word simplification and language modelling. We use a new dataset pairs of publicly available medical sentences and a version of them simplified by clinicians. Our method based on a language model trained on medical forum data generates simpler sentences while preserving both grammar and the original meaning.
  arXiv  Detail & Related papers  (2021-12-23T16:11:19Z)
• Semi-Supervised Variational Reasoning for Medical Dialogue Generation [70.838542865384]
  Two key characteristics are relevant for medical dialogue generation: patient states and physician actions. We propose an end-to-end variational reasoning approach to medical dialogue generation. A physician policy network composed of an action-classifier and two reasoning detectors is proposed for augmented reasoning ability.
  arXiv  Detail & Related papers  (2021-05-13T04:14:35Z)
• MedDG: An Entity-Centric Medical Consultation Dataset for Entity-Aware Medical Dialogue Generation [86.38736781043109]
  We build and release a large-scale high-quality Medical Dialogue dataset related to 12 types of common Gastrointestinal diseases named MedDG. We propose two kinds of medical dialogue tasks based on MedDG dataset. One is the next entity prediction and the other is the doctor response generation. Experimental results show that the pre-train language models and other baselines struggle on both tasks with poor performance in our dataset.
  arXiv  Detail & Related papers  (2020-10-15T03:34:33Z)
• BiteNet: Bidirectional Temporal Encoder Network to Predict Medical Outcomes [53.163089893876645]
  We propose a novel self-attention mechanism that captures the contextual dependency and temporal relationships within a patient's healthcare journey. An end-to-end bidirectional temporal encoder network (BiteNet) then learns representations of the patient's journeys. We have evaluated the effectiveness of our methods on two supervised prediction and two unsupervised clustering tasks with a real-world EHR dataset.
  arXiv  Detail & Related papers  (2020-09-24T00:42:36Z)
• Towards an Automated SOAP Note: Classifying Utterances from Medical Conversations [0.6875312133832078]
  We bridge the gap for classifying utterances from medical conversations according to (i) the SOAP section and (ii) the speaker role. We present a systematic analysis in which we adapt an existing deep learning architecture to the two aforementioned tasks. The results suggest that modelling context in a hierarchical manner, which captures both word and utterance level context, yields substantial improvements on both classification tasks.
  arXiv  Detail & Related papers  (2020-07-17T04:19:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.