Learning to Trim: End-to-End Causal Graph Pruning with Dynamic Anatomical Feature Banks for Medical VQA

• URL: http://arxiv.org/abs/2603.26028v1
• Date: Fri, 27 Mar 2026 02:57:39 GMT
• Title: Learning to Trim: End-to-End Causal Graph Pruning with Dynamic Anatomical Feature Banks for Medical VQA
• Abstract summary: We propose a Learnable Causal Trimming (LCT) framework that integrates causal pruning into end-to-end optimization.<n>We introduce a Dynamic Anatomical Feature Bank (DAFB), updated via a momentum mechanism, to capture global prototypes of frequent anatomical and linguistic patterns.<n>Experiments on VQA-RAD, SLAKE, SLAKE-CP and PathVQA demonstrate that LCT consistently improves robustness and generalization over existing debiasing strategies.
• Score: 27.877350391080693
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Medical Visual Question Answering (MedVQA) models often exhibit limited generalization due to reliance on dataset-specific correlations, such as recurring anatomical patterns or question-type regularities, rather than genuine diagnostic evidence. Existing causal approaches are typically implemented as static adjustments or post-hoc corrections. To address this issue, we propose a Learnable Causal Trimming (LCT) framework that integrates causal pruning into end-to-end optimization. We introduce a Dynamic Anatomical Feature Bank (DAFB), updated via a momentum mechanism, to capture global prototypes of frequent anatomical and linguistic patterns, serving as an approximation of dataset-level regularities. We further design a differentiable trimming module that estimates the dependency between instance-level representations and the global feature bank. Features highly correlated with global prototypes are softly suppressed, while instance-specific evidence is emphasized. This learnable mechanism encourages the model to prioritize causal signals over spurious correlations adaptively. Experiments on VQA-RAD, SLAKE, SLAKE-CP and PathVQA demonstrate that LCT consistently improves robustness and generalization over existing debiasing strategies.

Related papers

• Understanding Generalization through Decision Pattern Shift [46.36674063889302]
  Decision Pattern Shift (DPS) is a new perspective that defines generalization through the stability of internal decision patterns.<n>We show that DPS quantifies failure as their deviation from those learned during training.<n>These findings open up new possibilities for early generalization-risk detection, failure-mode diagnosis, and channel-level defect localization.
  arXiv  Detail & Related papers  (2026-05-13T08:14:59Z)
• MedCausalX: Adaptive Causal Reasoning with Self-Reflection for Trustworthy Medical Vision-Language Models [10.466505116993451]
  Existing medical chain-of-thought models lack explicit mechanisms to represent and enforce causal reasoning.<n>We propose MedCausalX, an end-to-end framework explicitly models causal reasoning chains in medical VLMs.<n>We show that MedCausalX consistently outperforms state-of-the-art methods, improving diagnostic consistency by +5.4 points, reducing hallucination by over 10 points, and attaining top spatial grounding IoU.
  arXiv  Detail & Related papers  (2026-03-24T11:28:15Z)
• Scalable Contrastive Causal Discovery under Unknown Soft Interventions [3.165716101116899]
  We propose a scalable causal discovery model for paired observational and interventional settings with shared underlying causal structure and unknown soft interventions.<n> Experiments on synthetic data demonstrate improved causal structure recovery, generalization to unseen graphs with held-out causal mechanisms, and scalability to larger graphs.
  arXiv  Detail & Related papers  (2026-03-03T18:16:16Z)
• A Generalized Adaptive Joint Learning Framework for High-Dimensional Time-Varying Models [0.8594140167290097]
  This article introduces Adaptive Joint Learning (AJL), a regularization framework designed to simultaneously perform functional variable selection and structural changepoint detection.<n>The analysis uncovers synchronized phase transitions in disease progression and identifies a parsimonious set of time-varying prognostic markers.
  arXiv  Detail & Related papers  (2026-01-08T02:07:49Z)
• Detecting and Pruning Prominent but Detrimental Neurons in Large Language Models [68.57424628540907]
  Large language models (LLMs) often develop learned mechanisms specialized to specific datasets.<n>We introduce a fine-tuning approach designed to enhance generalization by identifying and pruning neurons associated with dataset-specific mechanisms.<n>Our method employs Integrated Gradients to quantify each neuron's influence on high-confidence predictions, pinpointing those that disproportionately contribute to dataset-specific performance.
  arXiv  Detail & Related papers  (2025-07-12T08:10:10Z)
• Learning Time-Aware Causal Representation for Model Generalization in Evolving Domains [50.66049136093248]
  We develop a time-aware structural causal model (SCM) that incorporates dynamic causal factors and the causal mechanism drifts.<n>We show that our method can yield the optimal causal predictor for each time domain.<n>Results on both synthetic and real-world datasets exhibit that SYNC can achieve superior temporal generalization performance.
  arXiv  Detail & Related papers  (2025-06-21T14:05:37Z)
• Empowering Vision Transformers with Multi-Scale Causal Intervention for Long-Tailed Image Classification [12.122203089278738]
  Causal inference has emerged as a promising approach to mitigate long-tail classification by handling the biases introduced by class imbalance.<n>This paper investigates the influence of existing causal models on CNNs and ViT variants.<n>It proposes TSCNet, a two-stage causal modeling method to discover fine-grained causal associations.
  arXiv  Detail & Related papers  (2025-05-13T02:23:55Z)
• A Fixed-Point Approach for Causal Generative Modeling [20.88890689294816]
  We propose a novel formalism for describing Structural Causal Models (SCMs) as fixed-point problems on causally ordered variables.<n>We establish the weakest known conditions for their unique recovery given the topological ordering (TO)
  arXiv  Detail & Related papers  (2024-04-10T12:29:05Z)
• A PAC-Bayesian Perspective on the Interpolating Information Criterion [54.548058449535155]
  We show how a PAC-Bayes bound is obtained for a general class of models, characterizing factors which influence performance in the interpolating regime. We quantify how the test error for overparameterized models achieving effectively zero training error depends on the quality of the implicit regularization imposed by e.g. the combination of model, parameter-initialization scheme.
  arXiv  Detail & Related papers  (2023-11-13T01:48:08Z)
• ER: Equivariance Regularizer for Knowledge Graph Completion [107.51609402963072]
  We propose a new regularizer, namely, Equivariance Regularizer (ER) ER can enhance the generalization ability of the model by employing the semantic equivariance between the head and tail entities. The experimental results indicate a clear and substantial improvement over the state-of-the-art relation prediction methods.
  arXiv  Detail & Related papers  (2022-06-24T08:18:05Z)
• On the Generalization and Adaption Performance of Causal Models [99.64022680811281]
  Differentiable causal discovery has proposed to factorize the data generating process into a set of modules. We study the generalization and adaption performance of such modular neural causal models. Our analysis shows that the modular neural causal models outperform other models on both zero and few-shot adaptation in low data regimes.
  arXiv  Detail & Related papers  (2022-06-09T17:12:32Z)
• CASTLE: Regularization via Auxiliary Causal Graph Discovery [89.74800176981842]
  We introduce Causal Structure Learning (CASTLE) regularization and propose to regularize a neural network by jointly learning the causal relationships between variables. CASTLE efficiently reconstructs only the features in the causal DAG that have a causal neighbor, whereas reconstruction-based regularizers suboptimally reconstruct all input features.
  arXiv  Detail & Related papers  (2020-09-28T09:49:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.