Reason Like a Radiologist: Chain-of-Thought and Reinforcement Learning for Verifiable Report Generation

• URL: http://arxiv.org/abs/2504.18453v1
• Date: Fri, 25 Apr 2025 16:05:06 GMT
• Title: Reason Like a Radiologist: Chain-of-Thought and Reinforcement Learning for Verifiable Report Generation
• Authors: Peiyuan Jing, Kinhei Lee, Zhenxuan Zhang, Huichi Zhou, Zhengqing Yuan, Zhifan Gao, Lei Zhu, Giorgos Papanastasiou, Yingying Fang, Guang Yang,
• Abstract summary: BoxMed-RL is a groundbreaking unified training framework for generating spatially verifiable and explainable radiology reports.<n>Built on a large vision-language model, BoxMed-RL revolutionizes report generation through two integrated phases.<n>BoxMed-RL achieves an average 7% improvement in both METEOR and ROUGE-L metrics compared to state-of-the-art methods.
• Score: 13.580272788409092
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Radiology report generation is critical for efficiency but current models lack the structured reasoning of experts, hindering clinical trust and explainability by failing to link visual findings to precise anatomical locations. This paper introduces BoxMed-RL, a groundbreaking unified training framework for generating spatially verifiable and explainable radiology reports. Built on a large vision-language model, BoxMed-RL revolutionizes report generation through two integrated phases: (1) In the Pretraining Phase, we refine the model via medical concept learning, using Chain-of-Thought supervision to internalize the radiologist-like workflow, followed by spatially verifiable reinforcement, which applies reinforcement learning to align medical findings with bounding boxes. (2) In the Downstream Adapter Phase, we freeze the pretrained weights and train a downstream adapter to ensure fluent and clinically credible reports. This framework precisely mimics radiologists' workflow, compelling the model to connect high-level medical concepts with definitive anatomical evidence. Extensive experiments on public datasets demonstrate that BoxMed-RL achieves an average 7% improvement in both METEOR and ROUGE-L metrics compared to state-of-the-art methods. An average 5% improvement in large language model-based metrics further underscores BoxMed-RL's robustness in generating high-quality radiology reports.

Related papers

• Abn-BLIP: Abnormality-aligned Bootstrapping Language-Image Pre-training for Pulmonary Embolism Diagnosis and Report Generation from CTPA [3.1001390303501153]
  Abn-BLIP is an advanced diagnosis model designed to align abnormal findings to generate the accuracy and comprehensiveness of radiology reports.<n>Our experiments show that Abn-BLIP outperforms state-of-the-art medical vision-language models and 3D report generation methods in both accuracy and clinical relevance.
  arXiv  Detail & Related papers  (2025-03-03T20:13:39Z)
• Resource-Efficient Medical Report Generation using Large Language Models [3.2627279988912194]
  Medical report generation is the task of automatically writing radiology reports for chest X-ray images. We propose a new framework leveraging vision-enabled Large Language Models (LLM) for the task of medical report generation.
  arXiv  Detail & Related papers  (2024-10-21T05:08:18Z)
• TRRG: Towards Truthful Radiology Report Generation With Cross-modal Disease Clue Enhanced Large Language Model [22.305034251561835]
  We propose a truthful radiology report generation framework, namely TRRG, based on stage-wise training for cross-modal disease clue injection into large language models. Our proposed framework achieves state-of-the-art performance in radiology report generation on datasets such as IU-Xray and MIMIC-CXR.
  arXiv  Detail & Related papers  (2024-08-22T05:52:27Z)
• Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generation [14.86028303006519]
  We introduce a fact-aware multimodal retrieval-augmented pipeline in generating accurate radiology reports.<n>We first leverage RadGraph to mine factual report pairs, then integrate factual knowledge to train a universal multimodal retriever.<n>Experiments show that our multimodal retriever outperforms state-of-the-art retrievers on both language generation and radiology-specific metrics.
  arXiv  Detail & Related papers  (2024-07-21T21:04:28Z)
• Summarizing Radiology Reports Findings into Impressions [1.8964110318127383]
  We present a model with state-of-art radiology report summarization performance. We also provide an analysis of the model limitations and radiology knowledge gain. Our best performing model was a fine-tuned BERT-to-BERT encoder-decoder with 58.75/100 ROUGE-L F1.
  arXiv  Detail & Related papers  (2024-05-10T20:29:25Z)
• Beyond Images: An Integrative Multi-modal Approach to Chest X-Ray Report Generation [47.250147322130545]
  Image-to-text radiology report generation aims to automatically produce radiology reports that describe the findings in medical images. Most existing methods focus solely on the image data, disregarding the other patient information accessible to radiologists. We present a novel multi-modal deep neural network framework for generating chest X-rays reports by integrating structured patient data, such as vital signs and symptoms, alongside unstructured clinical notes.
  arXiv  Detail & Related papers  (2023-11-18T14:37:53Z)
• ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data [115.0747462486285]
  ChatRadio-Valuer is a tailored model for automatic radiology report generation that learns generalizable representations. The clinical dataset utilized in this study encompasses a remarkable total of textbf332,673 observations. ChatRadio-Valuer consistently outperforms state-of-the-art models, especially ChatGPT (GPT-3.5-Turbo) and GPT-4 et al.
  arXiv  Detail & Related papers  (2023-10-08T17:23:17Z)
• Radiology-Llama2: Best-in-Class Large Language Model for Radiology [71.27700230067168]
  This paper introduces Radiology-Llama2, a large language model specialized for radiology through a process known as instruction tuning. Quantitative evaluations using ROUGE metrics on the MIMIC-CXR and OpenI datasets demonstrate that Radiology-Llama2 achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-08-29T17:44:28Z)
• Customizing General-Purpose Foundation Models for Medical Report Generation [64.31265734687182]
  The scarcity of labelled medical image-report pairs presents great challenges in the development of deep and large-scale neural networks. We propose customizing off-the-shelf general-purpose large-scale pre-trained models, i.e., foundation models (FMs) in computer vision and natural language processing.
  arXiv  Detail & Related papers  (2023-06-09T03:02:36Z)
• Medical Image Captioning via Generative Pretrained Transformers [57.308920993032274]
  We combine two language models, the Show-Attend-Tell and the GPT-3, to generate comprehensive and descriptive radiology records. The proposed model is tested on two medical datasets, the Open-I, MIMIC-CXR, and the general-purpose MS-COCO.
  arXiv  Detail & Related papers  (2022-09-28T10:27:10Z)
• Radiology Report Generation with a Learned Knowledge Base and Multi-modal Alignment [27.111857943935725]
  We present an automatic, multi-modal approach for report generation from chest x-ray. Our approach features two distinct modules: (i) Learned knowledge base and (ii) Multi-modal alignment. With the aid of both modules, our approach clearly outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-12-30T10:43:56Z)
• MIMO: Mutual Integration of Patient Journey and Medical Ontology for Healthcare Representation Learning [49.57261599776167]
  We propose an end-to-end robust Transformer-based solution, Mutual Integration of patient journey and Medical Ontology (MIMO) for healthcare representation learning and predictive analytics.
  arXiv  Detail & Related papers  (2021-07-20T07:04:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.