Why Are You Weird? Infusing Interpretability in Isolation Forest for Anomaly Detection

• URL: http://arxiv.org/abs/2112.06858v1
• Date: Mon, 13 Dec 2021 18:16:18 GMT
• Title: Why Are You Weird? Infusing Interpretability in Isolation Forest for Anomaly Detection
• Authors: Nirmal Sobha Kartha, Cl\'ement Gautrais, and Vincent Vercruyssen
• Abstract summary: Anomaly detection is concerned with identifying examples in a dataset that do not conform to the expected behaviour. This paper develops a method to explain the anomaly predictions of the state-of-the-art Isolation Forest anomaly detection algorithm.
• Score: 3.498371632913735
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Anomaly detection is concerned with identifying examples in a dataset that do not conform to the expected behaviour. While a vast amount of anomaly detection algorithms exist, little attention has been paid to explaining why these algorithms flag certain examples as anomalies. However, such an explanation could be extremely useful to anyone interpreting the algorithms' output. This paper develops a method to explain the anomaly predictions of the state-of-the-art Isolation Forest anomaly detection algorithm. The method outputs an explanation vector that captures how important each attribute of an example is to identifying it as anomalous. A thorough experimental evaluation on both synthetic and real-world datasets shows that our method is more accurate and more efficient than most contemporary state-of-the-art explainability methods.

Related papers

• Can I trust my anomaly detection system? A case study based on explainable AI [0.4416503115535552]
  This case study explores the robustness of an anomaly detection system based on variational autoencoder generative models. The goal is to get a different perspective on the real performances of anomaly detectors that use reconstruction differences.
  arXiv  Detail & Related papers  (2024-07-29T12:39:07Z)
• xSemAD: Explainable Semantic Anomaly Detection in Event Logs Using Sequence-to-Sequence Models [1.6713531923053913]
  This work addresses a gap in semantic anomaly detection, which typically indicates the occurrence of an anomaly without explaining the nature of the anomaly. We propose xSemAD, an approach that uses a sequence-to-sequence model to go beyond pure identification and provides extended explanations. Our experiments demonstrate that our approach outperforms existing state-of-the-art semantic anomaly detection methods.
  arXiv  Detail & Related papers  (2024-06-28T09:06:52Z)
• Anomaly Detection by Context Contrasting [57.695202846009714]
  Anomaly detection focuses on identifying samples that deviate from the norm. Recent advances in self-supervised learning have shown great promise in this regard. We propose Con$$, which learns through context augmentations.
  arXiv  Detail & Related papers  (2024-05-29T07:59:06Z)
• Reimagining Anomalies: What If Anomalies Were Normal? [21.480869966442143]
  We introduce a novel explanation method that generates multiple counterfactual examples for each anomaly. A counterfactual example is a modification of the anomaly that is perceived as normal by the anomaly detector. The method provides a high-level semantic explanation of the mechanism that triggered the anomaly detector, allowing users to explore "what-if scenarios"
  arXiv  Detail & Related papers  (2024-02-22T11:56:44Z)
• Are we certain it's anomalous? [57.729669157989235]
  Anomaly detection in time series is a complex task since anomalies are rare due to highly non-linear temporal correlations. Here we propose the novel use of Hyperbolic uncertainty for Anomaly Detection (HypAD) HypAD learns self-supervisedly to reconstruct the input signal.
  arXiv  Detail & Related papers  (2022-11-16T21:31:39Z)
• Framing Algorithmic Recourse for Anomaly Detection [18.347886926848563]
  We present an approach -- Context preserving Algorithmic Recourse for Anomalies in Tabular data (CARAT) CARAT uses a transformer based encoder-decoder model to explain an anomaly by finding features with low likelihood. Semantically coherent counterfactuals are generated by modifying the highlighted features, using the overall context of features in the anomalous instance(s)
  arXiv  Detail & Related papers  (2022-06-29T03:30:51Z)
• Catching Both Gray and Black Swans: Open-set Supervised Anomaly Detection [90.32910087103744]
  A few labeled anomaly examples are often available in many real-world applications. These anomaly examples provide valuable knowledge about the application-specific abnormality. Those anomalies seen during training often do not illustrate every possible class of anomaly. This paper tackles open-set supervised anomaly detection.
  arXiv  Detail & Related papers  (2022-03-28T05:21:37Z)
• Explainable Deep Few-shot Anomaly Detection with Deviation Networks [123.46611927225963]
  We introduce a novel weakly-supervised anomaly detection framework to train detection models. The proposed approach learns discriminative normality by leveraging the labeled anomalies and a prior probability. Our model is substantially more sample-efficient and robust, and performs significantly better than state-of-the-art competing methods in both closed-set and open-set settings.
  arXiv  Detail & Related papers  (2021-08-01T14:33:17Z)
• Toward Deep Supervised Anomaly Detection: Reinforcement Learning from Partially Labeled Anomaly Data [150.9270911031327]
  We consider the problem of anomaly detection with a small set of partially labeled anomaly examples and a large-scale unlabeled dataset. Existing related methods either exclusively fit the limited anomaly examples that typically do not span the entire set of anomalies, or proceed with unsupervised learning from the unlabeled data. We propose here instead a deep reinforcement learning-based approach that enables an end-to-end optimization of the detection of both labeled and unlabeled anomalies.
  arXiv  Detail & Related papers  (2020-09-15T03:05:39Z)
• Interpretable Anomaly Detection with Mondrian P{\'o}lya Forests on Data Streams [6.177270420667713]
  Anomaly detection at scale is an extremely challenging problem of great practicality. Recent work has coalesced on variations of (random) $k$emphd-trees to summarise data for anomaly detection. These methods rely on ad-hoc score functions that are not easy to interpret. We contextualise these methods in a probabilistic framework which we call the Mondrian Polya Forest.
  arXiv  Detail & Related papers  (2020-08-04T13:19:07Z)
• Deep Weakly-supervised Anomaly Detection [118.55172352231381]
  Pairwise Relation prediction Network (PReNet) learns pairwise relation features and anomaly scores. PReNet can detect any seen/unseen abnormalities that fit the learned pairwise abnormal patterns. Empirical results on 12 real-world datasets show that PReNet significantly outperforms nine competing methods in detecting seen and unseen anomalies.
  arXiv  Detail & Related papers  (2019-10-30T00:40:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.