Differentiable Multi-Agent Actor-Critic for Multi-Step Radiology Report Summarization

• URL: http://arxiv.org/abs/2203.08257v1
• Date: Tue, 15 Mar 2022 21:18:09 GMT
• Title: Differentiable Multi-Agent Actor-Critic for Multi-Step Radiology Report Summarization
• Authors: Sanjeev Kumar Karn, Ning Liu, Hinrich Schuetze and Oladimeji Farri
• Abstract summary: The IMPRESSIONS section of a radiology report is a summary of the radiologist's reasoning and conclusions. Prior research on radiology report summarization has focused on single-step end-to-end models. We propose a two-step approach: extractive summarization followed by abstractive summarization.
• Score: 5.234281904315526
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The IMPRESSIONS section of a radiology report about an imaging study is a summary of the radiologist's reasoning and conclusions, and it also aids the referring physician in confirming or excluding certain diagnoses. A cascade of tasks are required to automatically generate an abstractive summary of the typical information-rich radiology report. These tasks include acquisition of salient content from the report and generation of a concise, easily consumable IMPRESSIONS section. Prior research on radiology report summarization has focused on single-step end-to-end models -- which subsume the task of salient content acquisition. To fully explore the cascade structure and explainability of radiology report summarization, we introduce two innovations. First, we design a two-step approach: extractive summarization followed by abstractive summarization. Second, we additionally break down the extractive part into two independent tasks: extraction of salient (1) sentences and (2) keywords. Experiments on a publicly available radiology report dataset show our novel approach leads to a more precise summary compared to single-step and to two-step-with-single-extractive-process baselines with an overall improvement in F1 score Of 3-4%.

Related papers

• RadBARTsum: Domain Specific Adaption of Denoising Sequence-to-Sequence Models for Abstractive Radiology Report Summarization [1.8450534779202723]
  This study proposes RadBARTsum, a domain-specific and facilitated adaptation of the BART model for abstractive radiology report summarization. The approach involves two main steps: 1) re-training the BART model on a large corpus of radiology reports using a novel entity masking strategy to improve biomedical domain knowledge learning, and 2) fine-tuning the model for the summarization task using the Findings and Background sections to predict the Impression section.
  arXiv  Detail & Related papers  (2024-06-05T08:43:11Z)
• Radiology Report Generation Using Transformers Conditioned with Non-imaging Data [55.17268696112258]
  This paper proposes a novel multi-modal transformer network that integrates chest x-ray (CXR) images and associated patient demographic information. The proposed network uses a convolutional neural network to extract visual features from CXRs and a transformer-based encoder-decoder network that combines the visual features with semantic text embeddings of patient demographic information.
  arXiv  Detail & Related papers  (2023-11-18T14:52:26Z)
• Controllable Chest X-Ray Report Generation from Longitudinal Representations [13.151444796296868]
  One strategy to speed up reporting is to integrate automated reporting systems. Previous approaches to automated radiology reporting generally do not provide the prior study as input. We introduce two novel aspects: (1) longitudinal learning -- we propose a method to align, leverage the current and prior scan information into a joint longitudinal representation which can be provided to the multimodal report generation model; (2) sentence-anatomy dropout -- a training strategy for controllability in which the report generator model is trained to predict only sentences from the original report which correspond to the subset of anatomical regions given as input.
  arXiv  Detail & Related papers  (2023-10-09T17:22:58Z)
• ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data [115.0747462486285]
  ChatRadio-Valuer is a tailored model for automatic radiology report generation that learns generalizable representations. The clinical dataset utilized in this study encompasses a remarkable total of textbf332,673 observations. ChatRadio-Valuer consistently outperforms state-of-the-art models, especially ChatGPT (GPT-3.5-Turbo) and GPT-4 et al.
  arXiv  Detail & Related papers  (2023-10-08T17:23:17Z)
• Parse and Recall: Towards Accurate Lung Nodule Malignancy Prediction like Radiologists [39.907916342786564]
  Lung cancer is a leading cause of death worldwide and early screening is critical for improving survival outcomes. In clinical practice, the contextual structure of nodules and the accumulated experience of radiologists are the two core elements related to the accuracy of identification of benign and malignant nodules. We propose a radiologist-inspired method to simulate the diagnostic process of radiologists, which is composed of context parsing and prototype recalling modules.
  arXiv  Detail & Related papers  (2023-07-20T12:38:17Z)
• Improving Radiology Summarization with Radiograph and Anatomy Prompts [60.30659124918211]
  We propose a novel anatomy-enhanced multimodal model to promote impression generation. In detail, we first construct a set of rules to extract anatomies and put these prompts into each sentence to highlight anatomy characteristics. We utilize a contrastive learning module to align these two representations at the overall level and use a co-attention to fuse them at the sentence level.
  arXiv  Detail & Related papers  (2022-10-15T14:05:03Z)
• Medical Image Captioning via Generative Pretrained Transformers [57.308920993032274]
  We combine two language models, the Show-Attend-Tell and the GPT-3, to generate comprehensive and descriptive radiology records. The proposed model is tested on two medical datasets, the Open-I, MIMIC-CXR, and the general-purpose MS-COCO.
  arXiv  Detail & Related papers  (2022-09-28T10:27:10Z)
• Factored Attention and Embedding for Unstructured-view Topic-related Ultrasound Report Generation [70.7778938191405]
  We propose a novel factored attention and embedding model (termed FAE-Gen) for the unstructured-view topic-related ultrasound report generation. The proposed FAE-Gen mainly consists of two modules, i.e., view-guided factored attention and topic-oriented factored embedding, which capture the homogeneous and heterogeneous morphological characteristic across different views.
  arXiv  Detail & Related papers  (2022-03-12T15:24:03Z)
• Learning Semi-Structured Representations of Radiology Reports [10.134080761449093]
  Given a corpus of radiology reports, researchers are often interested in identifying a subset of reports describing a particular medical finding. Recent studies proposed mapping free-text statements in radiology reports to semi-structured strings of terms taken from a limited vocabulary. This paper aims to present an approach for the automatic generation of semi-structured representations of radiology reports.
  arXiv  Detail & Related papers  (2021-12-20T18:53:41Z)
• Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generation [55.00308939833555]
  The PPKED includes three modules: Posterior Knowledge Explorer (PoKE), Prior Knowledge Explorer (PrKE) and Multi-domain Knowledge Distiller (MKD) PoKE explores the posterior knowledge, which provides explicit abnormal visual regions to alleviate visual data bias. PrKE explores the prior knowledge from the prior medical knowledge graph (medical knowledge) and prior radiology reports (working experience) to alleviate textual data bias.
  arXiv  Detail & Related papers  (2021-06-13T11:10:02Z)
• Show, Describe and Conclude: On Exploiting the Structure Information of Chest X-Ray Reports [5.6070625920019825]
  Chest X-Ray (CXR) images are commonly used for clinical screening and diagnosis. The complex structures between and within sections of the reports pose a great challenge to the automatic report generation. We propose a novel framework that exploits the structure information between and within report sections for generating CXR imaging reports.
  arXiv  Detail & Related papers  (2020-04-26T02:29:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.