A Classifier-Based Approach to Multi-Class Anomaly Detection for Astronomical Transients

• URL: http://arxiv.org/abs/2403.14742v1
• Date: Thu, 21 Mar 2024 18:00:00 GMT
• Title: A Classifier-Based Approach to Multi-Class Anomaly Detection for Astronomical Transients
• Authors: Rithwik Gupta, Daniel Muthukrishna, Michelle Lochner,
• Abstract summary: Real-time anomaly detection is essential for identifying rare transients in the era of large-scale astronomical surveys. Currently, most anomaly detection algorithms for astronomical transients rely on hand-crafted features extracted from light curves. We introduce an alternative approach to detecting anomalies: using the penultimate layer of a neural network classifier as the latent space for anomaly detection.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automating real-time anomaly detection is essential for identifying rare transients in the era of large-scale astronomical surveys. Modern survey telescopes are generating tens of thousands of alerts per night, and future telescopes, such as the Vera C. Rubin Observatory, are projected to increase this number dramatically. Currently, most anomaly detection algorithms for astronomical transients rely either on hand-crafted features extracted from light curves or on features generated through unsupervised representation learning, which are then coupled with standard machine learning anomaly detection algorithms. In this work, we introduce an alternative approach to detecting anomalies: using the penultimate layer of a neural network classifier as the latent space for anomaly detection. We then propose a novel method, named Multi-Class Isolation Forests (MCIF), which trains separate isolation forests for each class to derive an anomaly score for a light curve from the latent space representation given by the classifier. This approach significantly outperforms a standard isolation forest. We also use a simpler input method for real-time transient classifiers which circumvents the need for interpolation in light curves and helps the neural network model inter-passband relationships and handle irregular sampling. Our anomaly detection pipeline identifies rare classes including kilonovae, pair-instability supernovae, and intermediate luminosity transients shortly after trigger on simulated Zwicky Transient Facility light curves. Using a sample of our simulations that matched the population of anomalies expected in nature (54 anomalies and 12,040 common transients), our method was able to discover $41\pm3$ anomalies (~75% recall) after following up the top 2000 (~15%) ranked transients. Our novel method shows that classifiers can be effectively repurposed for real-time anomaly detection.

Related papers

• A Classifier-Based Approach to Multi-Class Anomaly Detection Applied to Astronomical Time-Series [0.0]
  anomaly detection is an open problem in many scientific fields. Most anomaly detection algorithms for astronomical time-series rely either on hand-crafted features or on features generated through unsupervised representation learning. We introduce a novel approach that leverages the latent space of a neural network classifier for anomaly detection.
  arXiv  Detail & Related papers  (2024-08-05T18:00:00Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• Video Anomaly Detection via Spatio-Temporal Pseudo-Anomaly Generation : A Unified Approach [49.995833831087175]
  This work proposes a novel method for generating generic Video-temporal PAs by inpainting a masked out region of an image. In addition, we present a simple unified framework to detect real-world anomalies under the OCC setting. Our method performs on par with other existing state-of-the-art PAs generation and reconstruction based methods under the OCC setting.
  arXiv  Detail & Related papers  (2023-11-27T13:14:06Z)
• CARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detection [53.83593870825628]
  One main challenge in time series anomaly detection (TSAD) is the lack of labelled data in many real-life scenarios. Most of the existing anomaly detection methods focus on learning the normal behaviour of unlabelled time series in an unsupervised manner. We introduce a novel end-to-end self-supervised ContrAstive Representation Learning approach for time series anomaly detection.
  arXiv  Detail & Related papers  (2023-08-18T04:45:56Z)
• Are we certain it's anomalous? [57.729669157989235]
  Anomaly detection in time series is a complex task since anomalies are rare due to highly non-linear temporal correlations. Here we propose the novel use of Hyperbolic uncertainty for Anomaly Detection (HypAD) HypAD learns self-supervisedly to reconstruct the input signal.
  arXiv  Detail & Related papers  (2022-11-16T21:31:39Z)
• Real-time Detection of Anomalies in Multivariate Time Series of Astronomical Data [0.0]
  Astronomical transients are stellar objects that become temporarily brighter on various timescales. New astronomical sky surveys are observing unprecedented numbers of multi-wavelength transients. We present two novel methods that aim to quickly and automatically detect anomalous transient light curves in real-time.
  arXiv  Detail & Related papers  (2021-12-15T19:02:54Z)
• Real-time detection of anomalies in large-scale transient surveys [0.0]
  We present two novel methods of automatically detecting anomalous transient light curves in real-time. Both methods are based on the simple idea that if the light curves from a known population of transients can be accurately modelled, any deviations from model predictions are likely anomalies.
  arXiv  Detail & Related papers  (2021-10-29T18:29:25Z)
• Explainable Deep Few-shot Anomaly Detection with Deviation Networks [123.46611927225963]
  We introduce a novel weakly-supervised anomaly detection framework to train detection models. The proposed approach learns discriminative normality by leveraging the labeled anomalies and a prior probability. Our model is substantially more sample-efficient and robust, and performs significantly better than state-of-the-art competing methods in both closed-set and open-set settings.
  arXiv  Detail & Related papers  (2021-08-01T14:33:17Z)
• PANDA : Perceptually Aware Neural Detection of Anomalies [20.838700258121197]
  We propose a novel fine-grained VAE-GAN architecture trained in a semi-supervised manner to detect both visually distinct and subtle anomalies. With the use of a residually connected dual-feature extractor, a fine-grained discriminator and a perceptual loss function, we are able to detect subtle, low inter-class (anomaly vs. normal) variant anomalies.
  arXiv  Detail & Related papers  (2021-04-28T11:03:50Z)
• A Deep Learning Approach for Active Anomaly Detection of Extragalactic Transients [1.7152709285783647]
  We present a variational recurrent autoencoder neural network to encode simulated Rubin Observatory extragalactic transient events. We rank 1,129,184 events based on an anomaly score estimated using an isolation forest. Our algorithm is able to identify these transients as anomalous well before peak, enabling real-time follow up studies.
  arXiv  Detail & Related papers  (2021-03-22T18:02:19Z)
• TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks [73.01104041298031]
  TadGAN is an unsupervised anomaly detection approach built on Generative Adversarial Networks (GANs) To capture the temporal correlations of time series, we use LSTM Recurrent Neural Networks as base models for Generators and Critics. To demonstrate the performance and generalizability of our approach, we test several anomaly scoring techniques and report the best-suited one.
  arXiv  Detail & Related papers  (2020-09-16T15:52:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.