Learning Under Adversarial and Interventional Shifts
- URL: http://arxiv.org/abs/2103.15933v1
- Date: Mon, 29 Mar 2021 20:10:51 GMT
- Title: Learning Under Adversarial and Interventional Shifts
- Authors: Harvineet Singh, Shalmali Joshi, Finale Doshi-Velez, Himabindu
Lakkaraju
- Abstract summary: We propose a new formulation, RISe, for designing robust models against a set of distribution shifts.
We employ the distributionally robust optimization framework to optimize the resulting objective in both supervised and reinforcement learning settings.
- Score: 36.183840774167756
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Machine learning models are often trained on data from one distribution and
deployed on others. So it becomes important to design models that are robust to
distribution shifts. Most of the existing work focuses on optimizing for either
adversarial shifts or interventional shifts. Adversarial methods lack
expressivity in representing plausible shifts as they consider shifts to joint
distributions in the data. Interventional methods allow more expressivity but
provide robustness to unbounded shifts, resulting in overly conservative
models. In this work, we combine the complementary strengths of the two
approaches and propose a new formulation, RISe, for designing robust models
against a set of distribution shifts that are at the intersection of
adversarial and interventional shifts. We employ the distributionally robust
optimization framework to optimize the resulting objective in both supervised
and reinforcement learning settings. Extensive experimentation with synthetic
and real world datasets from healthcare demonstrate the efficacy of the
proposed approach.
Related papers
- TransFusion: Covariate-Shift Robust Transfer Learning for High-Dimensional Regression [11.040033344386366]
We propose a two-step method with a novel fused-regularizer to improve the learning performance on a target task with limited samples.
Nonasymptotic bound is provided for the estimation error of the target model.
We extend the method to a distributed setting, allowing for a pretraining-finetuning strategy.
arXiv Detail & Related papers (2024-04-01T14:58:16Z) - InterHandGen: Two-Hand Interaction Generation via Cascaded Reverse Diffusion [53.90516061351706]
We present InterHandGen, a novel framework that learns the generative prior of two-hand interaction.
For sampling, we combine anti-penetration and synthesis-free guidance to enable plausible generation.
Our method significantly outperforms baseline generative models in terms of plausibility and diversity.
arXiv Detail & Related papers (2024-03-26T06:35:55Z) - Mitigating Shortcut Learning with Diffusion Counterfactuals and Diverse Ensembles [95.49699178874683]
We propose DiffDiv, an ensemble diversification framework exploiting Diffusion Probabilistic Models (DPMs)
We show that DPMs can generate images with novel feature combinations, even when trained on samples displaying correlated input features.
We show that DPM-guided diversification is sufficient to remove dependence on shortcut cues, without a need for additional supervised signals.
arXiv Detail & Related papers (2023-11-23T15:47:33Z) - Exact Subspace Diffusion for Decentralized Multitask Learning [17.592204922442832]
Distributed strategies for multitask learning induce relationships between agents in a more nuanced manner, and encourage collaboration without enforcing consensus.
We develop a generalization of the exact diffusion algorithm for subspace constrained multitask learning over networks, and derive an accurate expression for its mean-squared deviation.
We verify numerically the accuracy of the predicted performance expressions, as well as the improved performance of the proposed approach over alternatives based on approximate projections.
arXiv Detail & Related papers (2023-04-14T19:42:19Z) - Optimal Representations for Covariate Shift [18.136705088756138]
We introduce a simple variational objective whose optima are exactly the set of all representations on which risk minimizers are guaranteed to be robust.
Our objectives achieve state-of-the-art results on DomainBed, and give insights into the robustness of recent methods, such as CLIP.
arXiv Detail & Related papers (2021-12-31T21:02:24Z) - Towards Robust and Adaptive Motion Forecasting: A Causal Representation
Perspective [72.55093886515824]
We introduce a causal formalism of motion forecasting, which casts the problem as a dynamic process with three groups of latent variables.
We devise a modular architecture that factorizes the representations of invariant mechanisms and style confounders to approximate a causal graph.
Experiment results on synthetic and real datasets show that our three proposed components significantly improve the robustness and reusability of the learned motion representations.
arXiv Detail & Related papers (2021-11-29T18:59:09Z) - Encoding Robustness to Image Style via Adversarial Feature Perturbations [72.81911076841408]
We adapt adversarial training by directly perturbing feature statistics, rather than image pixels, to produce robust models.
Our proposed method, Adversarial Batch Normalization (AdvBN), is a single network layer that generates worst-case feature perturbations during training.
arXiv Detail & Related papers (2020-09-18T17:52:34Z) - Learning Diverse Representations for Fast Adaptation to Distribution
Shift [78.83747601814669]
We present a method for learning multiple models, incorporating an objective that pressures each to learn a distinct way to solve the task.
We demonstrate our framework's ability to facilitate rapid adaptation to distribution shift.
arXiv Detail & Related papers (2020-06-12T12:23:50Z) - Adversarial Distributional Training for Robust Deep Learning [53.300984501078126]
Adversarial training (AT) is among the most effective techniques to improve model robustness by augmenting training data with adversarial examples.
Most existing AT methods adopt a specific attack to craft adversarial examples, leading to the unreliable robustness against other unseen attacks.
In this paper, we introduce adversarial distributional training (ADT), a novel framework for learning robust models.
arXiv Detail & Related papers (2020-02-14T12:36:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.