GIT-CXR: End-to-End Transformer for Chest X-Ray Report Generation

• URL: http://arxiv.org/abs/2501.02598v1
• Date: Sun, 05 Jan 2025 16:45:49 GMT
• Title: GIT-CXR: End-to-End Transformer for Chest X-Ray Report Generation
• Authors: Iustin Sîrbu, Iulia-Renata Sîrbu, Jasmina Bogojeska, Traian Rebedea,
• Abstract summary: We have designed and evaluated an end-to-end transformer-based method to generate accurate and factually complete radiology reports for X-ray images. The experiments have been conducted using the MIMIC-CXR-JPG database, the largest available chest X-ray dataset.
• Score: 2.8900715468305767
• License:
• Abstract: Medical imaging is crucial for diagnosing, monitoring, and treating medical conditions. The medical reports of radiology images are the primary medium through which medical professionals attest their findings, but their writing is time consuming and requires specialized clinical expertise. The automated generation of radiography reports has thus the potential to improve and standardize patient care and significantly reduce clinicians workload. Through our work, we have designed and evaluated an end-to-end transformer-based method to generate accurate and factually complete radiology reports for X-ray images. Additionally, we are the first to introduce curriculum learning for end-to-end transformers in medical imaging and demonstrate its impact in obtaining improved performance. The experiments have been conducted using the MIMIC-CXR-JPG database, the largest available chest X-ray dataset. The results obtained are comparable with the current state-of-the-art on the natural language generation (NLG) metrics BLEU and ROUGE-L, while setting new state-of-the-art results on F1 examples-averaged, F1-macro and F1-micro metrics for clinical accuracy and on the METEOR metric widely used for NLG.

Related papers

• Resource-Efficient Medical Report Generation using Large Language Models [3.2627279988912194]
  Medical report generation is the task of automatically writing radiology reports for chest X-ray images. We propose a new framework leveraging vision-enabled Large Language Models (LLM) for the task of medical report generation.
  arXiv  Detail & Related papers  (2024-10-21T05:08:18Z)
• 3D-CT-GPT: Generating 3D Radiology Reports through Integration of Large Vision-Language Models [51.855377054763345]
  This paper introduces 3D-CT-GPT, a Visual Question Answering (VQA)-based medical visual language model for generating radiology reports from 3D CT scans. Experiments on both public and private datasets demonstrate that 3D-CT-GPT significantly outperforms existing methods in terms of report accuracy and quality.
  arXiv  Detail & Related papers  (2024-09-28T12:31:07Z)
• MiniGPT-Med: Large Language Model as a General Interface for Radiology Diagnosis [28.421857904824627]
  MiniGPT-Med is a vision-language model derived from large-scale language models and tailored for medical applications. It is capable of performing tasks such as medical report generation, visual question answering (VQA), and disease identification within medical imagery. It achieves state-of-the-art performance on medical report generation, higher than the previous best model by 19% accuracy.
  arXiv  Detail & Related papers  (2024-07-04T18:21:10Z)
• Large Model driven Radiology Report Generation with Clinical Quality Reinforcement Learning [16.849933628738277]
  Radiology report generation (RRG) has attracted significant attention due to its potential to reduce the workload of radiologists. This paper introduces a novel RRG method, textbfLM-RRG, that integrates large models (LMs) with clinical quality reinforcement learning. Experiments on the MIMIC-CXR and IU-Xray datasets demonstrate the superiority of our method over the state of the art.
  arXiv  Detail & Related papers  (2024-03-11T13:47:11Z)
• Radiology Report Generation Using Transformers Conditioned with Non-imaging Data [55.17268696112258]
  This paper proposes a novel multi-modal transformer network that integrates chest x-ray (CXR) images and associated patient demographic information. The proposed network uses a convolutional neural network to extract visual features from CXRs and a transformer-based encoder-decoder network that combines the visual features with semantic text embeddings of patient demographic information.
  arXiv  Detail & Related papers  (2023-11-18T14:52:26Z)
• ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data [115.0747462486285]
  ChatRadio-Valuer is a tailored model for automatic radiology report generation that learns generalizable representations. The clinical dataset utilized in this study encompasses a remarkable total of textbf332,673 observations. ChatRadio-Valuer consistently outperforms state-of-the-art models, especially ChatGPT (GPT-3.5-Turbo) and GPT-4 et al.
  arXiv  Detail & Related papers  (2023-10-08T17:23:17Z)
• Medical Image Captioning via Generative Pretrained Transformers [57.308920993032274]
  We combine two language models, the Show-Attend-Tell and the GPT-3, to generate comprehensive and descriptive radiology records. The proposed model is tested on two medical datasets, the Open-I, MIMIC-CXR, and the general-purpose MS-COCO.
  arXiv  Detail & Related papers  (2022-09-28T10:27:10Z)
• Cross-modal Clinical Graph Transformer for Ophthalmic Report Generation [116.87918100031153]
  We propose a Cross-modal clinical Graph Transformer (CGT) for ophthalmic report generation (ORG) CGT injects clinical relation triples into the visual features as prior knowledge to drive the decoding procedure. Experiments on the large-scale FFA-IR benchmark demonstrate that the proposed CGT is able to outperform previous benchmark methods.
  arXiv  Detail & Related papers  (2022-06-04T13:16:30Z)
• Cross-modal Memory Networks for Radiology Report Generation [30.13916304931662]
  Cross-modal memory networks (CMN) are proposed to enhance the encoder-decoder framework for radiology report generation. Our model is able to better align information from radiology images and texts so as to help generating more accurate reports in terms of clinical indicators.
  arXiv  Detail & Related papers  (2022-04-28T02:32:53Z)
• Medical-VLBERT: Medical Visual Language BERT for COVID-19 CT Report Generation With Alternate Learning [70.71564065885542]
  We propose to use the medical visual language BERT (Medical-VLBERT) model to identify the abnormality on the COVID-19 scans. This model adopts an alternate learning strategy with two procedures that are knowledge pretraining and transferring. For automatic medical report generation on the COVID-19 cases, we constructed a dataset of 368 medical findings in Chinese and 1104 chest CT scans.
  arXiv  Detail & Related papers  (2021-08-11T07:12:57Z)
• Generating Radiology Reports via Memory-driven Transformer [38.30011851429407]
  We propose to generate radiology reports with memory-driven Transformer. Experimental results on two prevailing radiology report datasets, IU X-Ray and MIMIC-CXR.
  arXiv  Detail & Related papers  (2020-10-30T04:08:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.