Distribution Shift Inversion for Out-of-Distribution Prediction
- URL: http://arxiv.org/abs/2306.08328v1
- Date: Wed, 14 Jun 2023 08:00:49 GMT
- Title: Distribution Shift Inversion for Out-of-Distribution Prediction
- Authors: Runpeng Yu, Songhua Liu, Xingyi Yang, Xinchao Wang
- Abstract summary: We propose a portable Distribution Shift Inversion algorithm for Out-of-Distribution (OoD) prediction.
We show that our method provides a general performance gain when plugged into a wide range of commonly used OoD algorithms.
- Score: 57.22301285120695
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Machine learning society has witnessed the emergence of a myriad of
Out-of-Distribution (OoD) algorithms, which address the distribution shift
between the training and the testing distribution by searching for a unified
predictor or invariant feature representation. However, the task of directly
mitigating the distribution shift in the unseen testing set is rarely
investigated, due to the unavailability of the testing distribution during the
training phase and thus the impossibility of training a distribution translator
mapping between the training and testing distribution. In this paper, we
explore how to bypass the requirement of testing distribution for distribution
translator training and make the distribution translation useful for OoD
prediction. We propose a portable Distribution Shift Inversion algorithm, in
which, before being fed into the prediction model, the OoD testing samples are
first linearly combined with additional Gaussian noise and then transferred
back towards the training distribution using a diffusion model trained only on
the source distribution. Theoretical analysis reveals the feasibility of our
method. Experimental results, on both multiple-domain generalization datasets
and single-domain generalization datasets, show that our method provides a
general performance gain when plugged into a wide range of commonly used OoD
algorithms.
Related papers
- Generalizing to any diverse distribution: uniformity, gentle finetuning and rebalancing [55.791818510796645]
We aim to develop models that generalize well to any diverse test distribution, even if the latter deviates significantly from the training data.
Various approaches like domain adaptation, domain generalization, and robust optimization attempt to address the out-of-distribution challenge.
We adopt a more conservative perspective by accounting for the worst-case error across all sufficiently diverse test distributions within a known domain.
arXiv Detail & Related papers (2024-10-08T12:26:48Z) - Any-Shift Prompting for Generalization over Distributions [66.29237565901734]
We propose any-shift prompting: a general probabilistic inference framework that considers the relationship between training and test distributions during prompt learning.
Within this framework, the test prompt exploits the distribution relationships to guide the generalization of the CLIP image-language model from training to any test distribution.
The network generates the tailored test prompt with both training and test information in a feedforward pass, avoiding extra training costs at test time.
arXiv Detail & Related papers (2024-02-15T16:53:42Z) - Invariant Anomaly Detection under Distribution Shifts: A Causal
Perspective [6.845698872290768]
Anomaly detection (AD) is the machine learning task of identifying highly discrepant abnormal samples.
Under the constraints of a distribution shift, the assumption that training samples and test samples are drawn from the same distribution breaks down.
We attempt to increase the resilience of anomaly detection models to different kinds of distribution shifts.
arXiv Detail & Related papers (2023-12-21T23:20:47Z) - Dr. FERMI: A Stochastic Distributionally Robust Fair Empirical Risk
Minimization Framework [12.734559823650887]
In the presence of distribution shifts, fair machine learning models may behave unfairly on test data.
Existing algorithms require full access to data and cannot be used when small batches are used.
This paper proposes the first distributionally robust fairness framework with convergence guarantees that do not require knowledge of the causal graph.
arXiv Detail & Related papers (2023-09-20T23:25:28Z) - Out-of-distribution Detection by Cross-class Vicinity Distribution of
In-distribution Data [36.66825830101456]
Deep neural networks for image classification only learn to map in-distribution inputs to their corresponding ground truth labels in training.
This results from the assumption that all samples are independent and identically distributed.
A textitCross-class Vicinity Distribution is introduced by assuming that an out-of-distribution sample generated by mixing multiple in-distribution samples does not share the same classes of its constituents.
arXiv Detail & Related papers (2022-06-19T12:03:33Z) - Robust Calibration with Multi-domain Temperature Scaling [86.07299013396059]
We develop a systematic calibration model to handle distribution shifts by leveraging data from multiple domains.
Our proposed method -- multi-domain temperature scaling -- uses the robustness in the domains to improve calibration under distribution shift.
arXiv Detail & Related papers (2022-06-06T17:32:12Z) - Distributional Reinforcement Learning via Moment Matching [54.16108052278444]
We formulate a method that learns a finite set of statistics from each return distribution via neural networks.
Our method can be interpreted as implicitly matching all orders of moments between a return distribution and its Bellman target.
Experiments on the suite of Atari games show that our method outperforms the standard distributional RL baselines.
arXiv Detail & Related papers (2020-07-24T05:18:17Z) - Calibrated Adversarial Refinement for Stochastic Semantic Segmentation [5.849736173068868]
We present a strategy for learning a calibrated predictive distribution over semantic maps, where the probability associated with each prediction reflects its ground truth correctness likelihood.
We demonstrate the versatility and robustness of the approach by achieving state-of-the-art results on the multigrader LIDC dataset and on a modified Cityscapes dataset with injected ambiguities.
We show that the core design can be adapted to other tasks requiring learning a calibrated predictive distribution by experimenting on a toy regression dataset.
arXiv Detail & Related papers (2020-06-23T16:39:59Z) - Balance-Subsampled Stable Prediction [55.13512328954456]
We propose a novel balance-subsampled stable prediction (BSSP) algorithm based on the theory of fractional factorial design.
A design-theoretic analysis shows that the proposed method can reduce the confounding effects among predictors induced by the distribution shift.
Numerical experiments on both synthetic and real-world data sets demonstrate that our BSSP algorithm significantly outperforms the baseline methods for stable prediction across unknown test data.
arXiv Detail & Related papers (2020-06-08T07:01:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.