Rethinking Radiology Report Generation via Causal Reasoning and
Counterfactual Augmentation
- URL: http://arxiv.org/abs/2311.13307v2
- Date: Tue, 5 Dec 2023 10:25:23 GMT
- Title: Rethinking Radiology Report Generation via Causal Reasoning and
Counterfactual Augmentation
- Authors: Xiao Song, Jiafan Liu, Yun Li, Wenbin Lei, Ruxin Wang
- Abstract summary: Radiology Report Generation (RRG) draws attention as an interaction between vision and language fields.
Joint Vision Coupling and Conditional Sentence Coherence Coupling are two aspects prone to implicitly decrease the accuracy of reports.
Counterfactual augmentation strategy that contains the Counterfactual Sample Synthesis and the Counterfactual Report Reconstruction sub-methods is proposed.
- Score: 8.138247389655675
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Radiology Report Generation (RRG) draws attention as an interaction between
vision and language fields. Previous works inherited the ideology of
vision-to-language generation tasks,aiming to generate paragraphs with high
consistency as reports. However, one unique characteristic of RRG, the
independence between diseases, was neglected, leading to the injection of
disease co-occurrence as a confounder that effects the results through backdoor
path. Unfortunately, this confounder confuses the process of report generation
worse because of the biased RRG data distribution. In this paper, to rethink
this issue thoroughly, we reason about its causes and effects from a novel
perspective of statistics and causality, where the Joint Vision Coupling and
the Conditional Sentence Coherence Coupling are two aspects prone to implicitly
decrease the accuracy of reports. Then, a counterfactual augmentation strategy
that contains the Counterfactual Sample Synthesis and the Counterfactual Report
Reconstruction sub-methods is proposed to break these two aspects of spurious
effects. Experimental results and further analyses on two widely used datasets
justify our reasoning and proposed methods.
Related papers
- Contrastive Learning with Counterfactual Explanations for Radiology Report Generation [83.30609465252441]
We propose a textbfCountertextbfFactual textbfExplanations-based framework (CoFE) for radiology report generation.
Counterfactual explanations serve as a potent tool for understanding how decisions made by algorithms can be changed by asking what if'' scenarios.
Experiments on two benchmarks demonstrate that leveraging the counterfactual explanations enables CoFE to generate semantically coherent and factually complete reports.
arXiv Detail & Related papers (2024-07-19T17:24:25Z) - Causality Extraction from Nuclear Licensee Event Reports Using a Hybrid Framework [3.1139106894905972]
This paper proposed a hybrid framework for causality detection and extraction from nuclear licensee event reports.
We compiled an LER corpus with 20,129 text samples for causality analysis, developed an interactive tool for labeling cause effect pairs, and built a deep-learning-based approach for causal relation detection.
arXiv Detail & Related papers (2024-04-08T16:39:34Z) - ICON: Improving Inter-Report Consistency of Radiology Report Generation
via Lesion-aware Mix-up Augmentation [15.342820385162709]
We propose ICON, which improves the inter-report consistency of radiology report generation.
Our approach involves first extracting lesions from input images and examining their characteristics.
Then, we introduce a lesion-aware mix-up augmentation technique to ensure that the representations of the semantically equivalent lesions align with the same attributes.
arXiv Detail & Related papers (2024-02-20T09:13:15Z) - Identifiable Latent Polynomial Causal Models Through the Lens of Change [85.67870425656368]
Causal representation learning aims to unveil latent high-level causal representations from observed low-level data.
One of its primary tasks is to provide reliable assurance of identifying these latent causal models, known as identifiability.
arXiv Detail & Related papers (2023-10-24T07:46:10Z) - Inducing Causal Structure for Abstractive Text Summarization [76.1000380429553]
We introduce a Structural Causal Model (SCM) to induce the underlying causal structure of the summarization data.
We propose a Causality Inspired Sequence-to-Sequence model (CI-Seq2Seq) to learn the causal representations that can mimic the causal factors.
Experimental results on two widely used text summarization datasets demonstrate the advantages of our approach.
arXiv Detail & Related papers (2023-08-24T16:06:36Z) - CausalDialogue: Modeling Utterance-level Causality in Conversations [83.03604651485327]
We have compiled and expanded upon a new dataset called CausalDialogue through crowd-sourcing.
This dataset includes multiple cause-effect pairs within a directed acyclic graph (DAG) structure.
We propose a causality-enhanced method called Exponential Average Treatment Effect (ExMATE) to enhance the impact of causality at the utterance level in training neural conversation models.
arXiv Detail & Related papers (2022-12-20T18:31:50Z) - Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation [3.3978173451092437]
Radiology report generation aims at generating descriptive text from radiology images automatically.
A typical setting consists of training encoder-decoder models on image-report pairs with a cross entropy loss.
We propose a novel weakly supervised contrastive loss for medical report generation.
arXiv Detail & Related papers (2021-09-25T00:06:23Z) - ACRE: Abstract Causal REasoning Beyond Covariation [90.99059920286484]
We introduce the Abstract Causal REasoning dataset for systematic evaluation of current vision systems in causal induction.
Motivated by the stream of research on causal discovery in Blicket experiments, we query a visual reasoning system with the following four types of questions in either an independent scenario or an interventional scenario.
We notice that pure neural models tend towards an associative strategy under their chance-level performance, whereas neuro-symbolic combinations struggle in backward-blocking reasoning.
arXiv Detail & Related papers (2021-03-26T02:42:38Z) - Improving Factual Completeness and Consistency of Image-to-Text
Radiology Report Generation [26.846912996765447]
We introduce two new simple rewards to encourage the generation of factually complete and consistent radiology reports.
We show that our system leads to generations that are more factually complete and consistent compared to the baselines.
arXiv Detail & Related papers (2020-10-20T05:42:47Z) - CausalVAE: Structured Causal Disentanglement in Variational Autoencoder [52.139696854386976]
The framework of variational autoencoder (VAE) is commonly used to disentangle independent factors from observations.
We propose a new VAE based framework named CausalVAE, which includes a Causal Layer to transform independent factors into causal endogenous ones.
Results show that the causal representations learned by CausalVAE are semantically interpretable, and their causal relationship as a Directed Acyclic Graph (DAG) is identified with good accuracy.
arXiv Detail & Related papers (2020-04-18T20:09:34Z) - Causal discovery of linear non-Gaussian acyclic models in the presence
of latent confounders [6.1221613913018675]
This paper proposes a causal functional model-based method called repetitive causal discovery (RCD) to discover the causal structure of observed variables affected by latent confounders.
RCD repeats inferring the causal directions between a small number of observed variables and determines whether the relationships are affected by latent confounders.
arXiv Detail & Related papers (2020-01-13T12:55:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.