Style-Aware Radiology Report Generation with RadGraph and Few-Shot Prompting

• URL: http://arxiv.org/abs/2310.17811v2
• Date: Tue, 31 Oct 2023 17:07:17 GMT
• Title: Style-Aware Radiology Report Generation with RadGraph and Few-Shot Prompting
• Authors: Benjamin Yan, Ruochen Liu, David E. Kuo, Subathra Adithan, Eduardo Pontes Reis, Stephen Kwak, Vasantha Kumar Venugopal, Chloe P. O'Connell, Agustina Saenz, Pranav Rajpurkar, Michael Moor
• Abstract summary: We propose a two-step approach for radiology report generation. First, we extract the content from an image; then, we verbalize the extracted content into a report that matches the style of a specific radiologist.
• Score: 5.596515201054671
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Automatically generated reports from medical images promise to improve the workflow of radiologists. Existing methods consider an image-to-report modeling task by directly generating a fully-fledged report from an image. However, this conflates the content of the report (e.g., findings and their attributes) with its style (e.g., format and choice of words), which can lead to clinically inaccurate reports. To address this, we propose a two-step approach for radiology report generation. First, we extract the content from an image; then, we verbalize the extracted content into a report that matches the style of a specific radiologist. For this, we leverage RadGraph -- a graph representation of reports -- together with large language models (LLMs). In our quantitative evaluations, we find that our approach leads to beneficial performance. Our human evaluation with clinical raters highlights that the AI-generated reports are indistinguishably tailored to the style of individual radiologist despite leveraging only a few examples as context.

Related papers

• MedRAT: Unpaired Medical Report Generation via Auxiliary Tasks [11.190146577567548]
  We propose a novel model that leverages the available information in two distinct datasets. Our model, named MedRAT, surpasses previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-04T13:31:47Z)
• MedCycle: Unpaired Medical Report Generation via Cycle-Consistency [11.190146577567548]
  We introduce an innovative approach that eliminates the need for consistent labeling schemas. This approach is based on cycle-consistent mapping functions that transform image embeddings into report embeddings. It outperforms state-of-the-art results in unpaired chest X-ray report generation, demonstrating improvements in both language and clinical metrics.
  arXiv  Detail & Related papers  (2024-03-20T09:40:11Z)
• RaDialog: A Large Vision-Language Model for Radiology Report Generation and Conversational Assistance [53.20640629352422]
  Conversational AI tools can generate and discuss clinically correct radiology reports for a given medical image. RaDialog is the first thoroughly evaluated and publicly available large vision-language model for radiology report generation and interactive dialog. Our method achieves state-of-the-art clinical correctness in report generation and shows impressive abilities in interactive tasks such as correcting reports and answering questions.
  arXiv  Detail & Related papers  (2023-11-30T16:28:40Z)
• Radiology Report Generation Using Transformers Conditioned with Non-imaging Data [55.17268696112258]
  This paper proposes a novel multi-modal transformer network that integrates chest x-ray (CXR) images and associated patient demographic information. The proposed network uses a convolutional neural network to extract visual features from CXRs and a transformer-based encoder-decoder network that combines the visual features with semantic text embeddings of patient demographic information.
  arXiv  Detail & Related papers  (2023-11-18T14:52:26Z)
• Beyond Images: An Integrative Multi-modal Approach to Chest X-Ray Report Generation [47.250147322130545]
  Image-to-text radiology report generation aims to automatically produce radiology reports that describe the findings in medical images. Most existing methods focus solely on the image data, disregarding the other patient information accessible to radiologists. We present a novel multi-modal deep neural network framework for generating chest X-rays reports by integrating structured patient data, such as vital signs and symptoms, alongside unstructured clinical notes.
  arXiv  Detail & Related papers  (2023-11-18T14:37:53Z)
• Multimodal Image-Text Matching Improves Retrieval-based Chest X-Ray Report Generation [3.6664023341224827]
  Contrastive X-Ray REport Match (X-REM) is a novel retrieval-based radiology report generation module. X-REM uses an image-text matching score to measure the similarity of a chest X-ray image and radiology report for report retrieval.
  arXiv  Detail & Related papers  (2023-03-29T04:00:47Z)
• Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generation [92.73584302508907]
  We propose a knowledge graph with Dynamic structure and nodes to facilitate medical report generation with Contrastive Learning. In detail, the fundamental structure of our graph is pre-constructed from general knowledge. Each image feature is integrated with its very own updated graph before being fed into the decoder module for report generation.
  arXiv  Detail & Related papers  (2023-03-18T03:53:43Z)
• Knowledge Graph Construction and Its Application in Automatic Radiology Report Generation from Radiologist's Dictation [22.894248859405767]
  This paper focuses on applications of NLP techniques like Information Extraction (IE) and domain-specific Knowledge Graph (KG) to automatically generate radiology reports from radiologist's dictation. We develop an information extraction pipeline that combines rule-based, pattern-based, and dictionary-based techniques with lexical-semantic features to extract entities and relations. We generate pathological descriptions evaluated using semantic similarity metrics, which shows 97% similarity with gold standard pathological descriptions.
  arXiv  Detail & Related papers  (2022-06-13T16:46:54Z)
• Variational Topic Inference for Chest X-Ray Report Generation [102.04931207504173]
  Report generation for medical imaging promises to reduce workload and assist diagnosis in clinical practice. Recent work has shown that deep learning models can successfully caption natural images. We propose variational topic inference for automatic report generation.
  arXiv  Detail & Related papers  (2021-07-15T13:34:38Z)
• Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generation [107.3538598876467]
  We propose an Auxiliary Signal-Guided Knowledge-Decoder (ASGK) to mimic radiologists' working patterns. ASGK integrates internal visual feature fusion and external medical linguistic information to guide medical knowledge transfer and learning.
  arXiv  Detail & Related papers  (2020-06-06T01:00:15Z)
• When Radiology Report Generation Meets Knowledge Graph [17.59749125131158]
  The accuracy of positive disease keyword mentions is critical in radiology image reporting. The evaluation of reporting quality should focus more on matching the disease keywords and their associated attributes. We propose a new evaluation metric for radiology image reporting with the assistance of the same composed graph.
  arXiv  Detail & Related papers  (2020-02-19T16:39:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.