Generation of Radiology Findings in Chest X-Ray by Leveraging
Collaborative Knowledge
- URL: http://arxiv.org/abs/2306.10448v1
- Date: Sun, 18 Jun 2023 00:51:28 GMT
- Title: Generation of Radiology Findings in Chest X-Ray by Leveraging
Collaborative Knowledge
- Authors: Manuela Daniela Danu, George Marica, Sanjeev Kumar Karn, Bogdan
Georgescu, Awais Mansoor, Florin Ghesu, Lucian Mihai Itu, Constantin Suciu,
Sasa Grbic, Oladimeji Farri, Dorin Comaniciu
- Abstract summary: The cognitive task of interpreting medical images remains the most critical and often time-consuming step in the radiology workflow.
This work focuses on reducing the workload of radiologists who spend most of their time either writing or narrating the Findings.
Unlike past research, which addresses radiology report generation as a single-step image captioning task, we have further taken into consideration the complexity of interpreting CXR images.
- Score: 6.792487817626456
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Among all the sub-sections in a typical radiology report, the Clinical
Indications, Findings, and Impression often reflect important details about the
health status of a patient. The information included in Impression is also
often covered in Findings. While Findings and Impression can be deduced by
inspecting the image, Clinical Indications often require additional context.
The cognitive task of interpreting medical images remains the most critical and
often time-consuming step in the radiology workflow. Instead of generating an
end-to-end radiology report, in this paper, we focus on generating the Findings
from automated interpretation of medical images, specifically chest X-rays
(CXRs). Thus, this work focuses on reducing the workload of radiologists who
spend most of their time either writing or narrating the Findings. Unlike past
research, which addresses radiology report generation as a single-step image
captioning task, we have further taken into consideration the complexity of
interpreting CXR images and propose a two-step approach: (a) detecting the
regions with abnormalities in the image, and (b) generating relevant text for
regions with abnormalities by employing a generative large language model
(LLM). This two-step approach introduces a layer of interpretability and aligns
the framework with the systematic reasoning that radiologists use when
reviewing a CXR.
Related papers
- Structural Entities Extraction and Patient Indications Incorporation for Chest X-ray Report Generation [10.46031380503486]
We introduce a novel method, textbfStructural textbfEntities extraction and patient indications textbfIncorporation (SEI) for chest X-ray report generation.
We employ a structural entities extraction (SEE) approach to eliminate presentation-style vocabulary in reports.
We propose a cross-modal fusion network to integrate information from X-ray images, similar historical cases, and patient-specific indications.
arXiv Detail & Related papers (2024-05-23T01:29:47Z) - Self-supervised vision-langage alignment of deep learning representations for bone X-rays analysis [53.809054774037214]
This paper proposes leveraging vision-language pretraining on bone X-rays paired with French reports.
It is the first study to integrate French reports to shape the embedding space devoted to bone X-Rays representations.
arXiv Detail & Related papers (2024-05-14T19:53:20Z) - Decoding Radiologists' Intentions: A Novel System for Accurate Region Identification in Chest X-ray Image Analysis [2.207061125661163]
In chest X-ray (CXR) image analysis, radiologists meticulously examine various regions, documenting their observations in reports.
The prevalence of errors in CXR diagnoses, particularly among inexperienced radiologists and hospital residents, underscores the importance of understanding radiologists' intentions and the corresponding regions of interest.
We propose a novel system designed to identify the primary intentions articulated by radiologists in their reports and the corresponding regions of interest in CXR images.
arXiv Detail & Related papers (2024-04-29T15:18:26Z) - Radiology Report Generation Using Transformers Conditioned with
Non-imaging Data [55.17268696112258]
This paper proposes a novel multi-modal transformer network that integrates chest x-ray (CXR) images and associated patient demographic information.
The proposed network uses a convolutional neural network to extract visual features from CXRs and a transformer-based encoder-decoder network that combines the visual features with semantic text embeddings of patient demographic information.
arXiv Detail & Related papers (2023-11-18T14:52:26Z) - Local Contrastive Learning for Medical Image Recognition [0.0]
Local Region Contrastive Learning (LRCLR) is a flexible fine-tuning framework that adds layers for significant image region selection and cross-modality interaction.
Our results on an external validation set of chest x-rays suggest that LRCLR identifies significant local image regions and provides meaningful interpretation against radiology text.
arXiv Detail & Related papers (2023-03-24T17:04:26Z) - Improving Radiology Summarization with Radiograph and Anatomy Prompts [60.30659124918211]
We propose a novel anatomy-enhanced multimodal model to promote impression generation.
In detail, we first construct a set of rules to extract anatomies and put these prompts into each sentence to highlight anatomy characteristics.
We utilize a contrastive learning module to align these two representations at the overall level and use a co-attention to fuse them at the sentence level.
arXiv Detail & Related papers (2022-10-15T14:05:03Z) - Medical Image Captioning via Generative Pretrained Transformers [57.308920993032274]
We combine two language models, the Show-Attend-Tell and the GPT-3, to generate comprehensive and descriptive radiology records.
The proposed model is tested on two medical datasets, the Open-I, MIMIC-CXR, and the general-purpose MS-COCO.
arXiv Detail & Related papers (2022-09-28T10:27:10Z) - Generative Residual Attention Network for Disease Detection [51.60842580044539]
We present a novel approach for disease generation in X-rays using a conditional generative adversarial learning.
We generate a corresponding radiology image in a target domain while preserving the identity of the patient.
We then use the generated X-ray image in the target domain to augment our training to improve the detection performance.
arXiv Detail & Related papers (2021-10-25T14:15:57Z) - Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report
Generation [107.3538598876467]
We propose an Auxiliary Signal-Guided Knowledge-Decoder (ASGK) to mimic radiologists' working patterns.
ASGK integrates internal visual feature fusion and external medical linguistic information to guide medical knowledge transfer and learning.
arXiv Detail & Related papers (2020-06-06T01:00:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.