Rethinking Radiology Report Generation via Causal Inspired Counterfactual Augmentation

• URL: http://arxiv.org/abs/2311.13307v3
• Date: Tue, 30 Jul 2024 16:11:48 GMT
• Title: Rethinking Radiology Report Generation via Causal Inspired Counterfactual Augmentation
• Authors: Xiao Song, Jiafan Liu, Yun Li, Yan Liu, Wenbin Lei, Ruxin Wang,
• Abstract summary: Radiology Report Generation (RRG) draws attention as a vision-and-language interaction of biomedical fields. Previous works inherited the ideology of traditional language generation tasks, aiming to generate paragraphs with high readability as reports. Despite significant progress, the independence between diseases-a specific property of RRG-was neglected, yielding the models being confused by the co-occurrence of diseases brought on by the biased data distribution.
• Score: 11.266364967223556
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Radiology Report Generation (RRG) draws attention as a vision-and-language interaction of biomedical fields. Previous works inherited the ideology of traditional language generation tasks, aiming to generate paragraphs with high readability as reports. Despite significant progress, the independence between diseases-a specific property of RRG-was neglected, yielding the models being confused by the co-occurrence of diseases brought on by the biased data distribution, thus generating inaccurate reports. In this paper, to rethink this issue, we first model the causal effects between the variables from a causal perspective, through which we prove that the co-occurrence relationships between diseases on the biased distribution function as confounders, confusing the accuracy through two backdoor paths, i.e. the Joint Vision Coupling and the Conditional Sequential Coupling. Then, we proposed a novel model-agnostic counterfactual augmentation method that contains two strategies, i.e. the Prototype-based Counterfactual Sample Synthesis (P-CSS) and the Magic-Cube-like Counterfactual Report Reconstruction (Cube), to intervene the backdoor paths, thus enhancing the accuracy and generalization of RRG models. Experimental results on the widely used MIMIC-CXR dataset demonstrate the effectiveness of our proposed method. Additionally, a generalization performance is evaluated on IU X-Ray dataset, which verifies our work can effectively reduce the impact of co-occurrences caused by different distributions on the results.

Related papers

• Targeted Cause Discovery with Data-Driven Learning [66.86881771339145]
  We propose a novel machine learning approach for inferring causal variables of a target variable from observations. We employ a neural network trained to identify causality through supervised learning on simulated data. Empirical results demonstrate the effectiveness of our method in identifying causal relationships within large-scale gene regulatory networks.
  arXiv  Detail & Related papers  (2024-08-29T02:21:11Z)
• ICON: Improving Inter-Report Consistency in Radiology Report Generation via Lesion-aware Mixup Augmentation [14.479606737135045]
  We propose ICON, which improves the inter-report consistency of radiology report generation. Our approach first involves extracting lesions from input images and examining their characteristics. Then, we introduce a lesion-aware mixup technique to ensure that the representations of the semantically equivalent lesions align with the same attributes.
  arXiv  Detail & Related papers  (2024-02-20T09:13:15Z)
• De-confounding Representation Learning for Counterfactual Inference on Continuous Treatment via Generative Adversarial Network [5.465397606401007]
  Counterfactual inference for continuous rather than binary treatment variables is more common in real-world causal inference tasks. We propose a de-confounding representation learning (DRL) framework for counterfactual outcome estimation of continuous treatment. We show that the DRL model performs superiorly in learning de-confounding representations and outperforms state-of-the-art counterfactual inference models for continuous treatment variables.
  arXiv  Detail & Related papers  (2023-07-24T08:56:25Z)
• Brain Imaging-to-Graph Generation using Adversarial Hierarchical Diffusion Models for MCI Causality Analysis [44.45598796591008]
  Brain imaging-to-graph generation (BIGG) framework is proposed to map functional magnetic resonance imaging (fMRI) into effective connectivity for mild cognitive impairment analysis. The hierarchical transformers in the generator are designed to estimate the noise at multiple scales. Evaluations of the ADNI dataset demonstrate the feasibility and efficacy of the proposed model.
  arXiv  Detail & Related papers  (2023-05-18T06:54:56Z)
• Cross-Modal Causal Intervention for Medical Report Generation [109.83549148448469]
  Medical report generation (MRG) is essential for computer-aided diagnosis and medication guidance. Due to the spurious correlations within image-text data induced by visual and linguistic biases, it is challenging to generate accurate reports reliably describing lesion areas. We propose a novel Visual-Linguistic Causal Intervention (VLCI) framework for MRG, which consists of a visual deconfounding module (VDM) and a linguistic deconfounding module (LDM)
  arXiv  Detail & Related papers  (2023-03-16T07:23:55Z)
• Less is More: Mitigate Spurious Correlations for Open-Domain Dialogue Response Generation Models by Causal Discovery [52.95935278819512]
  We conduct the first study on spurious correlations for open-domain response generation models based on a corpus CGDIALOG curated in our work. Inspired by causal discovery algorithms, we propose a novel model-agnostic method for training and inference of response generation model.
  arXiv  Detail & Related papers  (2023-03-02T06:33:48Z)
• BCDAG: An R package for Bayesian structure and Causal learning of Gaussian DAGs [77.34726150561087]
  We introduce the R package for causal discovery and causal effect estimation from observational data. Our implementation scales efficiently with the number of observations and, whenever the DAGs are sufficiently sparse, the number of variables in the dataset. We then illustrate the main functions and algorithms on both real and simulated datasets.
  arXiv  Detail & Related papers  (2022-01-28T09:30:32Z)
• Estimation of Bivariate Structural Causal Models by Variational Gaussian Process Regression Under Likelihoods Parametrised by Normalising Flows [74.85071867225533]
  Causal mechanisms can be described by structural causal models. One major drawback of state-of-the-art artificial intelligence is its lack of explainability.
  arXiv  Detail & Related papers  (2021-09-06T14:52:58Z)
• Harmonization with Flow-based Causal Inference [12.739380441313022]
  This paper presents a normalizing-flow-based method to perform counterfactual inference upon a structural causal model (SCM) to harmonize medical data. We evaluate on multiple, large, real-world medical datasets to observe that this method leads to better cross-domain generalization compared to state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2021-06-12T19:57:35Z)
• Efficient Causal Inference from Combined Observational and Interventional Data through Causal Reductions [68.6505592770171]
  Unobserved confounding is one of the main challenges when estimating causal effects. We propose a novel causal reduction method that replaces an arbitrary number of possibly high-dimensional latent confounders. We propose a learning algorithm to estimate the parameterized reduced model jointly from observational and interventional data.
  arXiv  Detail & Related papers  (2021-03-08T14:29:07Z)
• Causal discovery of linear non-Gaussian acyclic models in the presence of latent confounders [6.1221613913018675]
  This paper proposes a causal functional model-based method called repetitive causal discovery (RCD) to discover the causal structure of observed variables affected by latent confounders. RCD repeats inferring the causal directions between a small number of observed variables and determines whether the relationships are affected by latent confounders.
  arXiv  Detail & Related papers  (2020-01-13T12:55:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.