R2Gen-Mamba: A Selective State Space Model for Radiology Report Generation

• URL: http://arxiv.org/abs/2410.18135v1
• Date: Mon, 21 Oct 2024 19:35:34 GMT
• Title: R2Gen-Mamba: A Selective State Space Model for Radiology Report Generation
• Authors: Yongheng Sun, Yueh Z. Lee, Genevieve A. Woodard, Hongtu Zhu, Chunfeng Lian, Mingxia Liu,
• Abstract summary: R2Gen-Mamba is a novel automatic radiology report genera-tion method. R2Gen-Mamba not only enhances training and in-ference efficiency but also produces high-quality reports. Experimental results on two benchmark datasets with morethan 210,000 X-ray image-report pairs demonstrate the ef-fectiveness of R2Gen-Mamba.
• Score: 18.116802509376562
• License:
• Abstract: Radiology report generation is crucial in medical imaging,but the manual annotation process by physicians is time-consuming and labor-intensive, necessitating the develop-ment of automatic report generation methods. Existingresearch predominantly utilizes Transformers to generateradiology reports, which can be computationally intensive,limiting their use in real applications. In this work, we presentR2Gen-Mamba, a novel automatic radiology report genera-tion method that leverages the efficient sequence processingof the Mamba with the contextual benefits of Transformerarchitectures. Due to lower computational complexity ofMamba, R2Gen-Mamba not only enhances training and in-ference efficiency but also produces high-quality reports.Experimental results on two benchmark datasets with morethan 210,000 X-ray image-report pairs demonstrate the ef-fectiveness of R2Gen-Mamba regarding report quality andcomputational efficiency compared with several state-of-the-art methods. The source code can be accessed online.

Related papers

• Resource-Efficient Medical Report Generation using Large Language Models [3.2627279988912194]
  Medical report generation is the task of automatically writing radiology reports for chest X-ray images. We propose a new framework leveraging vision-enabled Large Language Models (LLM) for the task of medical report generation.
  arXiv  Detail & Related papers  (2024-10-21T05:08:18Z)
• R2GenCSR: Retrieving Context Samples for Large Language Model based X-ray Medical Report Generation [7.4871243017824165]
  This paper proposes a novel context-guided efficient X-ray medical report generation framework. Specifically, we introduce the Mamba as the vision backbone with linear complexity, and the performance obtained is comparable to that of the strong Transformer model.
  arXiv  Detail & Related papers  (2024-08-19T07:15:11Z)
• Prototype Learning Guided Hybrid Network for Breast Tumor Segmentation in DCE-MRI [58.809276442508256]
  We propose a hybrid network via the combination of convolution neural network (CNN) and transformer layers. The experimental results on private and public DCE-MRI datasets demonstrate that the proposed hybrid network superior performance than the state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-08-11T15:46:00Z)
• Radiology Report Generation Using Transformers Conditioned with Non-imaging Data [55.17268696112258]
  This paper proposes a novel multi-modal transformer network that integrates chest x-ray (CXR) images and associated patient demographic information. The proposed network uses a convolutional neural network to extract visual features from CXRs and a transformer-based encoder-decoder network that combines the visual features with semantic text embeddings of patient demographic information.
  arXiv  Detail & Related papers  (2023-11-18T14:52:26Z)
• PathLDM: Text conditioned Latent Diffusion Model for Histopathology [62.970593674481414]
  We introduce PathLDM, the first text-conditioned Latent Diffusion Model tailored for generating high-quality histopathology images. Our approach fuses image and textual data to enhance the generation process. We achieved a SoTA FID score of 7.64 for text-to-image generation on the TCGA-BRCA dataset, significantly outperforming the closest text-conditioned competitor with FID 30.1.
  arXiv  Detail & Related papers  (2023-09-01T22:08:32Z)
• Radiology-Llama2: Best-in-Class Large Language Model for Radiology [71.27700230067168]
  This paper introduces Radiology-Llama2, a large language model specialized for radiology through a process known as instruction tuning. Quantitative evaluations using ROUGE metrics on the MIMIC-CXR and OpenI datasets demonstrate that Radiology-Llama2 achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-08-29T17:44:28Z)
• KiUT: Knowledge-injected U-Transformer for Radiology Report Generation [10.139767157037829]
  Radiology report generation aims to automatically generate a clinically accurate and coherent paragraph from the X-ray image. We propose a Knowledge-injected U-Transformer (KiUT) to learn multi-level visual representation and adaptively distill the information.
  arXiv  Detail & Related papers  (2023-06-20T07:27:28Z)
• AlignTransformer: Hierarchical Alignment of Visual Regions and Disease Tags for Medical Report Generation [50.21065317817769]
  We propose an AlignTransformer framework, which includes the Align Hierarchical Attention (AHA) and the Multi-Grained Transformer (MGT) modules. Experiments on the public IU-Xray and MIMIC-CXR datasets show that the AlignTransformer can achieve results competitive with state-of-the-art methods on the two datasets.
  arXiv  Detail & Related papers  (2022-03-18T13:43:53Z)
• Generating Radiology Reports via Memory-driven Transformer [38.30011851429407]
  We propose to generate radiology reports with memory-driven Transformer. Experimental results on two prevailing radiology report datasets, IU X-Ray and MIMIC-CXR.
  arXiv  Detail & Related papers  (2020-10-30T04:08:03Z)
• Chest X-ray Report Generation through Fine-Grained Label Learning [46.352966049776875]
  We present a domain-aware automatic chest X-ray radiology report generation algorithm that learns fine-grained description of findings from images. We also develop an automatic labeling algorithm for assigning such descriptors to images and build a novel deep learning network that recognizes both coarse and fine-grained descriptions of findings.
  arXiv  Detail & Related papers  (2020-07-27T19:50:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.