Mixstyle-Entropy: Domain Generalization with Causal Intervention and Perturbation

• URL: http://arxiv.org/abs/2408.03608v2
• Date: Thu, 22 Aug 2024 13:13:56 GMT
• Title: Mixstyle-Entropy: Domain Generalization with Causal Intervention and Perturbation
• Authors: Luyao Tang, Yuxuan Yuan, Chaoqi Chen, Xinghao Ding, Yue Huang,
• Abstract summary: Domain generalization (DG) solves this issue by learning representations independent of domain-related information, thus facilitating extrapolation to unseen environments. Existing approaches typically focus on formulating tailored training objectives to extract shared features from the source data. We propose a novel framework based on causality, named InPer, designed to enhance model generalization by incorporating causal intervention during training and causal perturbation during testing.
• Score: 38.97031630265987
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the considerable advancements achieved by deep neural networks, their performance tends to degenerate when the test environment diverges from the training ones. Domain generalization (DG) solves this issue by learning representations independent of domain-related information, thus facilitating extrapolation to unseen environments. Existing approaches typically focus on formulating tailored training objectives to extract shared features from the source data. However, the disjointed training and testing procedures may compromise robustness, particularly in the face of unforeseen variations during deployment. In this paper, we propose a novel and holistic framework based on causality, named InPer, designed to enhance model generalization by incorporating causal intervention during training and causal perturbation during testing. Specifically, during the training phase, we employ entropy-based causal intervention (EnIn) to refine the selection of causal variables. To identify samples with anti-interference causal variables from the target domain, we propose a novel metric, homeostatic score, through causal perturbation (HoPer) to construct a prototype classifier in test time. Experimental results across multiple cross-domain tasks confirm the efficacy of InPer.

Related papers

• Targeted Cause Discovery with Data-Driven Learning [66.86881771339145]
  We propose a novel machine learning approach for inferring causal variables of a target variable from observations. We employ a neural network trained to identify causality through supervised learning on simulated data. Empirical results demonstrate the effectiveness of our method in identifying causal relationships within large-scale gene regulatory networks.
  arXiv  Detail & Related papers  (2024-08-29T02:21:11Z)
• Learning Unknown Intervention Targets in Structural Causal Models from Heterogeneous Data [27.38242139321935]
  We study the problem of identifying the unknown intervention targets in structural causal models. In the presence of latent confounders, the intervention targets among the observed variables cannot be determined uniquely. Our approach improves upon the state of the art as the returned candidate set is always a subset of the true intervention targets.
  arXiv  Detail & Related papers  (2023-12-11T03:31:54Z)
• Nonparametric Identifiability of Causal Representations from Unknown Interventions [63.1354734978244]
  We study causal representation learning, the task of inferring latent causal variables and their causal relations from mixtures of the variables. Our goal is to identify both the ground truth latents and their causal graph up to a set of ambiguities which we show to be irresolvable from interventional data.
  arXiv  Detail & Related papers  (2023-06-01T10:51:58Z)
• Deployment of Image Analysis Algorithms under Prevalence Shifts [6.373765910269204]
  Domain gaps are among the most relevant roadblocks in the clinical translation of machine learning (ML)-based solutions for medical image analysis. We propose a workflow for prevalence-aware image classification that uses estimated deployment prevalences to adjust a trained classifier to a new environment.
  arXiv  Detail & Related papers  (2023-03-22T13:16:37Z)
• Active Learning for Optimal Intervention Design in Causal Models [11.294389953686945]
  We develop a causal active learning strategy to identify interventions that are optimal, as measured by the discrepancy between the post-interventional mean of the distribution and a desired target mean. We apply our approach to both synthetic data and single-cell transcriptomic data from Perturb-CITE-seq experiments to identify optimal perturbations that induce a specific cell state transition.
  arXiv  Detail & Related papers  (2022-09-10T20:40:30Z)
• DRFLM: Distributionally Robust Federated Learning with Inter-client Noise via Local Mixup [58.894901088797376]
  federated learning has emerged as a promising approach for training a global model using data from multiple organizations without leaking their raw data. We propose a general framework to solve the above two challenges simultaneously. We provide comprehensive theoretical analysis including robustness analysis, convergence analysis, and generalization ability.
  arXiv  Detail & Related papers  (2022-04-16T08:08:29Z)
• Pre-training also Transfers Non-Robustness [20.226917627173126]
  In spite of its recognized contribution to generalization, pre-training also transfers the non-robustness from pre-trained model into the fine-tuned model. Results validate the effectiveness in alleviating non-robustness and preserving generalization.
  arXiv  Detail & Related papers  (2021-06-21T11:16:13Z)
• Decentralized Local Stochastic Extra-Gradient for Variational Inequalities [125.62877849447729]
  We consider distributed variational inequalities (VIs) on domains with the problem data that is heterogeneous (non-IID) and distributed across many devices. We make a very general assumption on the computational network that covers the settings of fully decentralized calculations. We theoretically analyze its convergence rate in the strongly-monotone, monotone, and non-monotone settings.
  arXiv  Detail & Related papers  (2021-06-15T17:45:51Z)
• Unsupervised Domain Adaptation for Spatio-Temporal Action Localization [69.12982544509427]
  S-temporal action localization is an important problem in computer vision. We propose an end-to-end unsupervised domain adaptation algorithm. We show that significant performance gain can be achieved when spatial and temporal features are adapted separately or jointly.
  arXiv  Detail & Related papers  (2020-10-19T04:25:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.