ORID: Organ-Regional Information Driven Framework for Radiology Report Generation

• URL: http://arxiv.org/abs/2411.13025v1
• Date: Wed, 20 Nov 2024 04:13:43 GMT
• Title: ORID: Organ-Regional Information Driven Framework for Radiology Report Generation
• Abstract summary: The objective of Radiology Report Generation (RRG) is to automatically generate coherent textual analyses of diseases based on radiological images. Current AI-based methods for RRG primarily focus on modifications to the encoder-decoder model architecture. This paper introduces an Organ-Regional Information Driven (ORID) framework which can effectively integrate multi-modal information.
• Score: 14.47027682763605
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The objective of Radiology Report Generation (RRG) is to automatically generate coherent textual analyses of diseases based on radiological images, thereby alleviating the workload of radiologists. Current AI-based methods for RRG primarily focus on modifications to the encoder-decoder model architecture. To advance these approaches, this paper introduces an Organ-Regional Information Driven (ORID) framework which can effectively integrate multi-modal information and reduce the influence of noise from unrelated organs. Specifically, based on the LLaVA-Med, we first construct an RRG-related instruction dataset to improve organ-regional diagnosis description ability and get the LLaVA-Med-RRG. After that, we propose an organ-based cross-modal fusion module to effectively combine the information from the organ-regional diagnosis description and radiology image. To further reduce the influence of noise from unrelated organs on the radiology report generation, we introduce an organ importance coefficient analysis module, which leverages Graph Neural Network (GNN) to examine the interconnections of the cross-modal information of each organ region. Extensive experiments an1d comparisons with state-of-the-art methods across various evaluation metrics demonstrate the superior performance of our proposed method.

Related papers

• CT-GRAPH: Hierarchical Graph Attention Network for Anatomy-Guided CT Report Generation [4.376648893167674]
  We propose CT-GRAPH, a hierarchical graph attention network that explicitly models radiological knowledge.<n>Our method leverages pretrained 3D medical feature encoders to obtain global and organ-level features.<n>We show that our method achieves a substantial improvement of absolute 7.9% in F1 score over current state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-08-07T13:18:03Z)
• FG-CXR: A Radiologist-Aligned Gaze Dataset for Enhancing Interpretability in Chest X-Ray Report Generation [9.374812942790953]
  We introduce Fine-Grained CXR dataset, which provides fine-grained paired information between the captions generated by radiologists and the corresponding gaze attention heatmaps for each anatomy. Our analysis reveals that simply applying black-box image captioning methods to generate reports cannot adequately explain which information in CXR is utilized. We propose a novel explainable radiologist's attention generator network (Gen-XAI) that mimics the diagnosis process of radiologists, explicitly constraining its output to closely align with both radiologist's gaze attention and transcript.
  arXiv  Detail & Related papers  (2024-11-23T02:22:40Z)
• Uncovering Knowledge Gaps in Radiology Report Generation Models through Knowledge Graphs [18.025481751074214]
  We introduce a system, named ReXKG, which extracts structured information from processed reports to construct a radiology knowledge graph. We conduct an in-depth comparative analysis of AI-generated and human-written radiology reports, assessing the performance of both specialist and generalist models.
  arXiv  Detail & Related papers  (2024-08-26T16:28:56Z)
• AutoRG-Brain: Grounded Report Generation for Brain MRI [57.22149878985624]
  Radiologists are tasked with interpreting a large number of images in a daily base, with the responsibility of generating corresponding reports. This demanding workload elevates the risk of human error, potentially leading to treatment delays, increased healthcare costs, revenue loss, and operational inefficiencies. We initiate a series of work on grounded Automatic Report Generation (AutoRG) This system supports the delineation of brain structures, the localization of anomalies, and the generation of well-organized findings.
  arXiv  Detail & Related papers  (2024-07-23T17:50:00Z)
• Large Model driven Radiology Report Generation with Clinical Quality Reinforcement Learning [16.849933628738277]
  Radiology report generation (RRG) has attracted significant attention due to its potential to reduce the workload of radiologists. This paper introduces a novel RRG method, textbfLM-RRG, that integrates large models (LMs) with clinical quality reinforcement learning. Experiments on the MIMIC-CXR and IU-Xray datasets demonstrate the superiority of our method over the state of the art.
  arXiv  Detail & Related papers  (2024-03-11T13:47:11Z)
• ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data [115.0747462486285]
  ChatRadio-Valuer is a tailored model for automatic radiology report generation that learns generalizable representations. The clinical dataset utilized in this study encompasses a remarkable total of textbf332,673 observations. ChatRadio-Valuer consistently outperforms state-of-the-art models, especially ChatGPT (GPT-3.5-Turbo) and GPT-4 et al.
  arXiv  Detail & Related papers  (2023-10-08T17:23:17Z)
• Radiology-Llama2: Best-in-Class Large Language Model for Radiology [71.27700230067168]
  This paper introduces Radiology-Llama2, a large language model specialized for radiology through a process known as instruction tuning. Quantitative evaluations using ROUGE metrics on the MIMIC-CXR and OpenI datasets demonstrate that Radiology-Llama2 achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-08-29T17:44:28Z)
• Cross-Modal Causal Intervention for Medical Report Generation [109.83549148448469]
  Medical report generation (MRG) is essential for computer-aided diagnosis and medication guidance. Due to the spurious correlations within image-text data induced by visual and linguistic biases, it is challenging to generate accurate reports reliably describing lesion areas. We propose a novel Visual-Linguistic Causal Intervention (VLCI) framework for MRG, which consists of a visual deconfounding module (VDM) and a linguistic deconfounding module (LDM)
  arXiv  Detail & Related papers  (2023-03-16T07:23:55Z)
• Self adaptive global-local feature enhancement for radiology report generation [10.958641951927817]
  We propose a novel framework AGFNet to dynamically fuse the global and anatomy region feature to generate multi-grained radiology report. Firstly, we extract important anatomy region features and global features of input Chest X-ray (CXR) Then, with the region features and the global features as input, our proposed self-adaptive fusion gate module could dynamically fuse multi-granularity information. Finally, the captioning generator generates the radiology reports through multi-granularity features.
  arXiv  Detail & Related papers  (2022-11-21T11:50:42Z)
• A unified 3D framework for Organs at Risk Localization and Segmentation for Radiation Therapy Planning [56.52933974838905]
  Current medical workflow requires manual delineation of organs-at-risk (OAR) In this work, we aim to introduce a unified 3D pipeline for OAR localization-segmentation. Our proposed framework fully enables the exploitation of 3D context information inherent in medical imaging.
  arXiv  Detail & Related papers  (2022-03-01T17:08:41Z)
• Generative Residual Attention Network for Disease Detection [51.60842580044539]
  We present a novel approach for disease generation in X-rays using a conditional generative adversarial learning. We generate a corresponding radiology image in a target domain while preserving the identity of the patient. We then use the generated X-ray image in the target domain to augment our training to improve the detection performance.
  arXiv  Detail & Related papers  (2021-10-25T14:15:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.