Invariant Anomaly Detection under Distribution Shifts: A Causal
Perspective
- URL: http://arxiv.org/abs/2312.14329v1
- Date: Thu, 21 Dec 2023 23:20:47 GMT
- Title: Invariant Anomaly Detection under Distribution Shifts: A Causal
Perspective
- Authors: Jo\~ao B. S. Carvalho, Mengtao Zhang, Robin Geyer, Carlos Cotrini,
Joachim M. Buhmann
- Abstract summary: Anomaly detection (AD) is the machine learning task of identifying highly discrepant abnormal samples.
Under the constraints of a distribution shift, the assumption that training samples and test samples are drawn from the same distribution breaks down.
We attempt to increase the resilience of anomaly detection models to different kinds of distribution shifts.
- Score: 6.845698872290768
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Anomaly detection (AD) is the machine learning task of identifying highly
discrepant abnormal samples by solely relying on the consistency of the normal
training samples. Under the constraints of a distribution shift, the assumption
that training samples and test samples are drawn from the same distribution
breaks down. In this work, by leveraging tools from causal inference we attempt
to increase the resilience of anomaly detection models to different kinds of
distribution shifts. We begin by elucidating a simple yet necessary statistical
property that ensures invariant representations, which is critical for robust
AD under both domain and covariate shifts. From this property, we derive a
regularization term which, when minimized, leads to partial distribution
invariance across environments. Through extensive experimental evaluation on
both synthetic and real-world tasks, covering a range of six different AD
methods, we demonstrated significant improvements in out-of-distribution
performance. Under both covariate and domain shift, models regularized with our
proposed term showed marked increased robustness. Code is available at:
https://github.com/JoaoCarv/invariant-anomaly-detection.
Related papers
- Proxy Methods for Domain Adaptation [78.03254010884783]
proxy variables allow for adaptation to distribution shift without explicitly recovering or modeling latent variables.
We develop a two-stage kernel estimation approach to adapt to complex distribution shifts in both settings.
arXiv Detail & Related papers (2024-03-12T09:32:41Z) - Training Implicit Generative Models via an Invariant Statistical Loss [3.139474253994318]
Implicit generative models have the capability to learn arbitrary complex data distributions.
On the downside, training requires telling apart real data from artificially-generated ones using adversarial discriminators.
We develop a discriminator-free method for training one-dimensional (1D) generative implicit models.
arXiv Detail & Related papers (2024-02-26T09:32:28Z) - Supervised Contrastive Learning with Heterogeneous Similarity for
Distribution Shifts [3.7819322027528113]
We propose a new regularization using the supervised contrastive learning to prevent such overfitting and to train models that do not degrade their performance under the distribution shifts.
Experiments on benchmark datasets that emulate distribution shifts, including subpopulation shift and domain generalization, demonstrate the advantage of the proposed method.
arXiv Detail & Related papers (2023-04-07T01:45:09Z) - Robust Calibration with Multi-domain Temperature Scaling [86.07299013396059]
We develop a systematic calibration model to handle distribution shifts by leveraging data from multiple domains.
Our proposed method -- multi-domain temperature scaling -- uses the robustness in the domains to improve calibration under distribution shift.
arXiv Detail & Related papers (2022-06-06T17:32:12Z) - Certifying Model Accuracy under Distribution Shifts [151.67113334248464]
We present provable robustness guarantees on the accuracy of a model under bounded Wasserstein shifts of the data distribution.
We show that a simple procedure that randomizes the input of the model within a transformation space is provably robust to distributional shifts under the transformation.
arXiv Detail & Related papers (2022-01-28T22:03:50Z) - Covariate Shift in High-Dimensional Random Feature Regression [44.13449065077103]
Covariate shift is a significant obstacle in the development of robust machine learning models.
We present a theoretical understanding in context of modern machine learning.
arXiv Detail & Related papers (2021-11-16T05:23:28Z) - Predicting with Confidence on Unseen Distributions [90.68414180153897]
We connect domain adaptation and predictive uncertainty literature to predict model accuracy on challenging unseen distributions.
We find that the difference of confidences (DoC) of a classifier's predictions successfully estimates the classifier's performance change over a variety of shifts.
We specifically investigate the distinction between synthetic and natural distribution shifts and observe that despite its simplicity DoC consistently outperforms other quantifications of distributional difference.
arXiv Detail & Related papers (2021-07-07T15:50:18Z) - Unsupervised Anomaly Detection From Semantic Similarity Scores [0.0]
We present a simple and generic framework, it SemSAD, that makes use of a semantic similarity score to carry out anomaly detection.
We are able to outperform previous approaches for anomaly, novelty, or out-of-distribution detection in the visual domain by a large margin.
arXiv Detail & Related papers (2020-12-01T13:12:31Z) - GANs with Variational Entropy Regularizers: Applications in Mitigating
the Mode-Collapse Issue [95.23775347605923]
Building on the success of deep learning, Generative Adversarial Networks (GANs) provide a modern approach to learn a probability distribution from observed samples.
GANs often suffer from the mode collapse issue where the generator fails to capture all existing modes of the input distribution.
We take an information-theoretic approach and maximize a variational lower bound on the entropy of the generated samples to increase their diversity.
arXiv Detail & Related papers (2020-09-24T19:34:37Z) - Estimating Generalization under Distribution Shifts via Domain-Invariant
Representations [75.74928159249225]
We use a set of domain-invariant predictors as a proxy for the unknown, true target labels.
The error of the resulting risk estimate depends on the target risk of the proxy model.
arXiv Detail & Related papers (2020-07-06T17:21:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.