ICON: Improving Inter-Report Consistency in Radiology Report Generation via Lesion-aware Mixup Augmentation

• URL: http://arxiv.org/abs/2402.12844v2
• Date: Thu, 26 Sep 2024 11:29:04 GMT
• Title: ICON: Improving Inter-Report Consistency in Radiology Report Generation via Lesion-aware Mixup Augmentation
• Authors: Wenjun Hou, Yi Cheng, Kaishuai Xu, Yan Hu, Wenjie Li, Jiang Liu,
• Abstract summary: We propose ICON, which improves the inter-report consistency of radiology report generation. Our approach first involves extracting lesions from input images and examining their characteristics. Then, we introduce a lesion-aware mixup technique to ensure that the representations of the semantically equivalent lesions align with the same attributes.
• Score: 14.479606737135045
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Previous research on radiology report generation has made significant progress in terms of increasing the clinical accuracy of generated reports. In this paper, we emphasize another crucial quality that it should possess, i.e., inter-report consistency, which refers to the capability of generating consistent reports for semantically equivalent radiographs. This quality is even of greater significance than the overall report accuracy in terms of ensuring the system's credibility, as a system prone to providing conflicting results would severely erode users' trust. Regrettably, existing approaches struggle to maintain inter-report consistency, exhibiting biases towards common patterns and susceptibility to lesion variants. To address this issue, we propose ICON, which improves the inter-report consistency of radiology report generation. Aiming to enhance the system's ability to capture similarities in semantically equivalent lesions, our approach first involves extracting lesions from input images and examining their characteristics. Then, we introduce a lesion-aware mixup technique to ensure that the representations of the semantically equivalent lesions align with the same attributes, achieved through a linear combination during the training phase. Extensive experiments on three publicly available chest X-ray datasets verify the effectiveness of our approach, both in terms of improving the consistency and accuracy of the generated reports.

Related papers

• Contrastive Learning with Counterfactual Explanations for Radiology Report Generation [83.30609465252441]
  We propose a textbfCountertextbfFactual textbfExplanations-based framework (CoFE) for radiology report generation. Counterfactual explanations serve as a potent tool for understanding how decisions made by algorithms can be changed by asking what if'' scenarios. Experiments on two benchmarks demonstrate that leveraging the counterfactual explanations enables CoFE to generate semantically coherent and factually complete reports.
  arXiv  Detail & Related papers  (2024-07-19T17:24:25Z)
• Synchronous Faithfulness Monitoring for Trustworthy Retrieval-Augmented Generation [96.78845113346809]
  Retrieval-augmented language models (RALMs) have shown strong performance and wide applicability in knowledge-intensive tasks. This paper proposes SynCheck, a lightweight monitor that leverages fine-grained decoding dynamics to detect unfaithful sentences. We also introduce FOD, a faithfulness-oriented decoding algorithm guided by beam search for long-form retrieval-augmented generation.
  arXiv  Detail & Related papers  (2024-06-19T16:42:57Z)
• Rethinking Radiology Report Generation via Causal Inspired Counterfactual Augmentation [11.266364967223556]
  Radiology Report Generation (RRG) draws attention as a vision-and-language interaction of biomedical fields. Previous works inherited the ideology of traditional language generation tasks, aiming to generate paragraphs with high readability as reports. Despite significant progress, the independence between diseases-a specific property of RRG-was neglected, yielding the models being confused by the co-occurrence of diseases brought on by the biased data distribution.
  arXiv  Detail & Related papers  (2023-11-22T10:55:36Z)
• Beyond Images: An Integrative Multi-modal Approach to Chest X-Ray Report Generation [47.250147322130545]
  Image-to-text radiology report generation aims to automatically produce radiology reports that describe the findings in medical images. Most existing methods focus solely on the image data, disregarding the other patient information accessible to radiologists. We present a novel multi-modal deep neural network framework for generating chest X-rays reports by integrating structured patient data, such as vital signs and symptoms, alongside unstructured clinical notes.
  arXiv  Detail & Related papers  (2023-11-18T14:37:53Z)
• Towards Reliable Medical Image Segmentation by utilizing Evidential Calibrated Uncertainty [52.03490691733464]
  We introduce DEviS, an easily implementable foundational model that seamlessly integrates into various medical image segmentation networks. By leveraging subjective logic theory, we explicitly model probability and uncertainty for the problem of medical image segmentation. DeviS incorporates an uncertainty-aware filtering module, which utilizes the metric of uncertainty-calibrated error to filter reliable data.
  arXiv  Detail & Related papers  (2023-01-01T05:02:46Z)
• Fairness and robustness in anti-causal prediction [73.693135253335]
  Robustness to distribution shift and fairness have independently emerged as two important desiderata required of machine learning models. While these two desiderata seem related, the connection between them is often unclear in practice. By taking this perspective, we draw explicit connections between a common fairness criterion - separation - and a common notion of robustness.
  arXiv  Detail & Related papers  (2022-09-20T02:41:17Z)
• A Self-Guided Framework for Radiology Report Generation [10.573538773141715]
  A self-guided framework (SGF) is developed to generate medical reports with annotated disease labels. SGF uses unsupervised and supervised deep learning methods to mimic the process of human learning and writing. Our results highlight the capacity of the proposed framework to distinguish fined-grained visual details between words.
  arXiv  Detail & Related papers  (2022-06-19T11:09:27Z)
• Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation [3.3978173451092437]
  Radiology report generation aims at generating descriptive text from radiology images automatically. A typical setting consists of training encoder-decoder models on image-report pairs with a cross entropy loss. We propose a novel weakly supervised contrastive loss for medical report generation.
  arXiv  Detail & Related papers  (2021-09-25T00:06:23Z)
• Confidence-Guided Radiology Report Generation [24.714303916431078]
  We propose a novel method to quantify both the visual uncertainty and the textual uncertainty for the task of radiology report generation. Our experimental results have demonstrated that our proposed method for model uncertainty characterization and estimation can provide more reliable confidence scores for radiology report generation.
  arXiv  Detail & Related papers  (2021-06-21T07:02:12Z)
• Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation [26.846912996765447]
  We introduce two new simple rewards to encourage the generation of factually complete and consistent radiology reports. We show that our system leads to generations that are more factually complete and consistent compared to the baselines.
  arXiv  Detail & Related papers  (2020-10-20T05:42:47Z)
• Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model [71.80319052891817]
  We present a relation-driven semi-supervised framework for medical image classification. It exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations. Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.
  arXiv  Detail & Related papers  (2020-05-15T06:57:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.