Effectiveness of ChatGPT in explaining complex medical reports to patients

• URL: http://arxiv.org/abs/2406.15963v1
• Date: Sun, 23 Jun 2024 00:04:07 GMT
• Title: Effectiveness of ChatGPT in explaining complex medical reports to patients
• Authors: Mengxuan Sun, Ehud Reiter, Anne E Kiltie, George Ramsay, Lisa Duncan, Peter Murchie, Rosalind Adam,
• Abstract summary: We explore whether ChatGPT (GPT 4) can help explain multidisciplinary team (MDT) reports to colorectal and prostate cancer patients. These reports are written in dense medical language and assume clinical knowledge, so they are a good test of the ability of ChatGPT to explain complex medical reports to patients.
• Score: 3.8249624199073553
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Electronic health records contain detailed information about the medical condition of patients, but they are difficult for patients to understand even if they have access to them. We explore whether ChatGPT (GPT 4) can help explain multidisciplinary team (MDT) reports to colorectal and prostate cancer patients. These reports are written in dense medical language and assume clinical knowledge, so they are a good test of the ability of ChatGPT to explain complex medical reports to patients. We asked clinicians and lay people (not patients) to review explanations and responses of ChatGPT. We also ran three focus groups (including cancer patients, caregivers, computer scientists, and clinicians) to discuss output of ChatGPT. Our studies highlighted issues with inaccurate information, inappropriate language, limited personalization, AI distrust, and challenges integrating large language models (LLMs) into clinical workflow. These issues will need to be resolved before LLMs can be used to explain complex personal medical information to patients.

Related papers

• CataractBot: An LLM-Powered Expert-in-the-Loop Chatbot for Cataract Patients [5.649965979758816]
  CataractBot was developed in collaboration with an eye hospital in India. It answers cataract surgery related questions instantly by querying a curated knowledge base. Users reported that their trust in the system was established through expert verification.
  arXiv  Detail & Related papers  (2024-02-07T07:07:02Z)
• Impact of Large Language Model Assistance on Patients Reading Clinical Notes: A Mixed-Methods Study [46.5728291706842]
  We developed a patient-facing tool using large language models (LLMs) to make clinical notes more readable. We piloted the tool with clinical notes donated by patients with a history of breast cancer and synthetic notes from a clinician.
  arXiv  Detail & Related papers  (2024-01-17T23:14:52Z)
• Yes, this is what I was looking for! Towards Multi-modal Medical Consultation Concern Summary Generation [46.42604861624895]
  We propose a new task of multi-modal medical concern summary generation. Nonverbal cues, such as patients' gestures and facial expressions, aid in accurately identifying patients' concerns. We construct the first multi-modal medical concern summary generation corpus.
  arXiv  Detail & Related papers  (2024-01-10T12:56:47Z)
• Language models are susceptible to incorrect patient self-diagnosis in medical applications [0.0]
  We present a variety of LLMs with multiple-choice questions from U.S. medical board exams modified to include self-diagnostic reports from patients. Our findings highlight that when a patient proposes incorrect bias-validating information, the diagnostic accuracy of LLMs drop dramatically.
  arXiv  Detail & Related papers  (2023-09-17T19:56:39Z)
• Challenges of GPT-3-based Conversational Agents for Healthcare [11.517862889784293]
  This paper investigates the challenges and risks of using GPT-3-based models for medical question-answering (MedQA) We provide a procedure for manually designing patient queries to stress-test high-risk limitations of LLMs in MedQA systems. Our analysis reveals that LLMs fail to respond adequately to these queries, generating erroneous medical information, unsafe recommendations, and content that may be considered offensive.
  arXiv  Detail & Related papers  (2023-08-28T15:12:34Z)
• MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records [60.35217378132709]
  Large language models (LLMs) can follow natural language instructions with human-level fluency. evaluating LLMs on realistic text generation tasks for healthcare remains challenging. We introduce MedAlign, a benchmark dataset of 983 natural language instructions for EHR data.
  arXiv  Detail & Related papers  (2023-08-27T12:24:39Z)
• Generating medically-accurate summaries of patient-provider dialogue: A multi-stage approach using large language models [6.252236971703546]
  An effective summary is required to be coherent and accurately capture all the medically relevant information in the dialogue. This paper tackles the problem of medical conversation summarization by discretizing the task into several smaller dialogue-understanding tasks.
  arXiv  Detail & Related papers  (2023-05-10T08:48:53Z)
• ChatGPT Makes Medicine Easy to Swallow: An Exploratory Case Study on Simplified Radiology Reports [0.4194454151396506]
  ChatGPT is a language model capable of generating text that appears human-like and authentic. We asked 15 radiologists to assess the quality of radiology reports simplified by ChatGPT. Most radiologists agreed that the simplified reports were factually correct, complete, and not potentially harmful to the patient.
  arXiv  Detail & Related papers  (2022-12-30T18:55:16Z)
• Retrieval-Augmented and Knowledge-Grounded Language Models for Faithful Clinical Medicine [68.7814360102644]
  We propose the Re$3$Writer method with retrieval-augmented generation and knowledge-grounded reasoning. We demonstrate the effectiveness of our method in generating patient discharge instructions.
  arXiv  Detail & Related papers  (2022-10-23T16:34:39Z)
• VBridge: Connecting the Dots Between Features, Explanations, and Data for Healthcare Models [85.4333256782337]
  VBridge is a visual analytics tool that seamlessly incorporates machine learning explanations into clinicians' decision-making workflow. We identified three key challenges, including clinicians' unfamiliarity with ML features, lack of contextual information, and the need for cohort-level evidence. We demonstrated the effectiveness of VBridge through two case studies and expert interviews with four clinicians.
  arXiv  Detail & Related papers  (2021-08-04T17:34:13Z)
• On the Generation of Medical Dialogues for COVID-19 [60.63485429268256]
  People experiencing COVID19-related symptoms or exposed to risk factors have a pressing need to consult doctors. Because of the shortage of medical professionals, many people cannot receive online consultations timely. We aim to develop a medical dialogue system that can provide COVID19-related consultations.
  arXiv  Detail & Related papers  (2020-05-11T21:23:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.