No Free Lunch: The Hazards of Over-Expressive Representations in Anomaly Detection

• URL: http://arxiv.org/abs/2306.07284v1
• Date: Mon, 12 Jun 2023 17:59:50 GMT
• Title: No Free Lunch: The Hazards of Over-Expressive Representations in Anomaly Detection
• Authors: Tal Reiss, Niv Cohen, Yedid Hoshen
• Abstract summary: We show that state-of-the-art representations often suffer from over-expressivity, failing to detect many types of anomalies. Our investigation demonstrates how this over-expressivity impairs image anomaly detection in practical settings.
• Score: 35.128547933798274
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Anomaly detection methods, powered by deep learning, have recently been making significant progress, mostly due to improved representations. It is tempting to hypothesize that anomaly detection can improve indefinitely by increasing the scale of our networks, making their representations more expressive. In this paper, we provide theoretical and empirical evidence to the contrary. In fact, we empirically show cases where very expressive representations fail to detect even simple anomalies when evaluated beyond the well-studied object-centric datasets. To investigate this phenomenon, we begin by introducing a novel theoretical toy model for anomaly detection performance. The model uncovers a fundamental trade-off between representation sufficiency and over-expressivity. It provides evidence for a no-free-lunch theorem in anomaly detection stating that increasing representation expressivity will eventually result in performance degradation. Instead, guidance must be provided to focus the representation on the attributes relevant to the anomalies of interest. We conduct an extensive empirical investigation demonstrating that state-of-the-art representations often suffer from over-expressivity, failing to detect many types of anomalies. Our investigation demonstrates how this over-expressivity impairs image anomaly detection in practical settings. We conclude with future directions for mitigating this issue.

Related papers

• Class-wise Activation Unravelling the Engima of Deep Double Descent [0.0]
  Double descent presents a counter-intuitive aspect within the machine learning domain. In this study, we revisited the phenomenon of double descent and discussed the conditions of its occurrence.
  arXiv  Detail & Related papers  (2024-05-13T12:07:48Z)
• Diffusion Model with Cross Attention as an Inductive Bias for Disentanglement [58.9768112704998]
  Disentangled representation learning strives to extract the intrinsic factors within observed data. We introduce a new perspective and framework, demonstrating that diffusion models with cross-attention can serve as a powerful inductive bias. This is the first work to reveal the potent disentanglement capability of diffusion models with cross-attention, requiring no complex designs.
  arXiv  Detail & Related papers  (2024-02-15T05:07:54Z)
• CARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detection [53.83593870825628]
  One main challenge in time series anomaly detection (TSAD) is the lack of labelled data in many real-life scenarios. Most of the existing anomaly detection methods focus on learning the normal behaviour of unlabelled time series in an unsupervised manner. We introduce a novel end-to-end self-supervised ContrAstive Representation Learning approach for time series anomaly detection.
  arXiv  Detail & Related papers  (2023-08-18T04:45:56Z)
• Updated version: A Video Anomaly Detection Framework based on Appearance-Motion Semantics Representation Consistency [2.395616571632115]
  We propose a framework of Appearance-Motion Semantics Consistency Representation. The two-stream structure is designed to encode the appearance and motion information representation of normal samples. A novel consistency loss is proposed to enhance the consistency of feature semantics so that anomalies with low consistency can be identified.
  arXiv  Detail & Related papers  (2023-03-09T08:28:34Z)
• Anomaly Detection Requires Better Representations [28.611440466398715]
  Anomaly detection seeks to identify unusual phenomena, a central task in science and industry. Recent advances in self-supervised representation learning have driven improvements in anomaly detection.
  arXiv  Detail & Related papers  (2022-10-19T17:59:32Z)
• The Familiarity Hypothesis: Explaining the Behavior of Deep Open Set Methods [86.39044549664189]
  Anomaly detection algorithms for feature-vector data identify anomalies as outliers, but outlier detection has not worked well in deep learning. This paper proposes the Familiarity Hypothesis that these methods succeed because they are detecting the absence of familiar learned features rather than the presence of novelty. The paper concludes with a discussion of whether familiarity detection is an inevitable consequence of representation learning.
  arXiv  Detail & Related papers  (2022-03-04T18:32:58Z)
• Explainable Deep Few-shot Anomaly Detection with Deviation Networks [123.46611927225963]
  We introduce a novel weakly-supervised anomaly detection framework to train detection models. The proposed approach learns discriminative normality by leveraging the labeled anomalies and a prior probability. Our model is substantially more sample-efficient and robust, and performs significantly better than state-of-the-art competing methods in both closed-set and open-set settings.
  arXiv  Detail & Related papers  (2021-08-01T14:33:17Z)
• Towards Fair Deep Anomaly Detection [24.237000220172906]
  We propose a new architecture for the fair anomaly detection approach (Deep Fair SVDD) We show that our proposed approach can remove the unfairness largely with minimal loss on the anomaly detection performance.
  arXiv  Detail & Related papers  (2020-12-29T22:34:45Z)
• Manifolds for Unsupervised Visual Anomaly Detection [79.22051549519989]
  Unsupervised learning methods that don't necessarily encounter anomalies in training would be immensely useful. We develop a novel hyperspherical Variational Auto-Encoder (VAE) via stereographic projections with a gyroplane layer. We present state-of-the-art results on visual anomaly benchmarks in precision manufacturing and inspection, demonstrating real-world utility in industrial AI scenarios.
  arXiv  Detail & Related papers  (2020-06-19T20:41:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.