Novelty Detection on Radio Astronomy Data using Signatures

• URL: http://arxiv.org/abs/2402.14892v2
• Date: Tue, 12 Mar 2024 17:34:16 GMT
• Title: Novelty Detection on Radio Astronomy Data using Signatures
• Authors: Paola Arrubarrena, Maud Lemercier, Bojan Nikolic, Terry Lyons, Thomas Cass
• Abstract summary: We introduce SigNova, a new semi-supervised framework for detecting anomalies in streamed data. We use the signature transform to extract a canonical collection of statistics from observational sequences. Each feature vector is assigned a novelty score, calculated as the Mahalanobis distance to its nearest neighbor in an RFI-free training set.
• Score: 5.304803553490439
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce SigNova, a new semi-supervised framework for detecting anomalies in streamed data. While our initial examples focus on detecting radio-frequency interference (RFI) in digitized signals within the field of radio astronomy, it is important to note that SigNova's applicability extends to any type of streamed data. The framework comprises three primary components. Firstly, we use the signature transform to extract a canonical collection of summary statistics from observational sequences. This allows us to represent variable-length visibility samples as finite-dimensional feature vectors. Secondly, each feature vector is assigned a novelty score, calculated as the Mahalanobis distance to its nearest neighbor in an RFI-free training set. By thresholding these scores we identify observation ranges that deviate from the expected behavior of RFI-free visibility samples without relying on stringent distributional assumptions. Thirdly, we integrate this anomaly detector with Pysegments, a segmentation algorithm, to localize consecutive observations contaminated with RFI, if any. This approach provides a compelling alternative to classical windowing techniques commonly used for RFI detection. Importantly, the complexity of our algorithm depends on the RFI pattern rather than on the size of the observation window. We demonstrate how SigNova improves the detection of various types of RFI (e.g., broadband and narrowband) in time-frequency visibility data. We validate our framework on the Murchison Widefield Array (MWA) telescope and simulated data and the Hydrogen Epoch of Reionization Array (HERA).

Related papers

• PeFAD: A Parameter-Efficient Federated Framework for Time Series Anomaly Detection [51.20479454379662]
  We propose a. Federated Anomaly Detection framework named PeFAD with the increasing privacy concerns. We conduct extensive evaluations on four real datasets, where PeFAD outperforms existing state-of-the-art baselines by up to 28.74%.
  arXiv  Detail & Related papers  (2024-06-04T13:51:08Z)
• Imbalanced Aircraft Data Anomaly Detection [103.01418862972564]
  Anomaly detection in temporal data from sensors under aviation scenarios is a practical but challenging task. We propose a Graphical Temporal Data Analysis framework. It consists three modules, named Series-to-Image (S2I), Cluster-based Resampling Approach using Euclidean Distance (CRD) and Variance-Based Loss (VBL)
  arXiv  Detail & Related papers  (2023-05-17T09:37:07Z)
• HiNoVa: A Novel Open-Set Detection Method for Automating RF Device Authentication [9.571774189070531]
  We introduce a novel open-set detection approach based on the patterns of the hidden state values within a Convolutional Neural Network (CNN) Long Short-Term Memory (LSTM) model. Our approach greatly improves the Area Under the Precision-Recall Curve on LoRa, Wireless-WiFi, and Wired-WiFi datasets.
  arXiv  Detail & Related papers  (2023-05-16T16:47:02Z)
• LSTM and CNN application for core-collapse supernova search in gravitational wave real data [0.0]
  Core-collapse supernovae (CCSNe) are expected to emit gravitational wave signals that could be detected by interferometers within the Milky Way and nearby galaxies. We show potential of machine learning (ML) for multi-label classification of different CCSNe simulated signals and noise transients using real data.
  arXiv  Detail & Related papers  (2023-01-23T12:12:33Z)
• Boosting 3D Object Detection by Simulating Multimodality on Point Clouds [51.87740119160152]
  This paper presents a new approach to boost a single-modality (LiDAR) 3D object detector by teaching it to simulate features and responses that follow a multi-modality (LiDAR-image) detector. The approach needs LiDAR-image data only when training the single-modality detector, and once well-trained, it only needs LiDAR data at inference. Experimental results on the nuScenes dataset show that our approach outperforms all SOTA LiDAR-only 3D detectors.
  arXiv  Detail & Related papers  (2022-06-30T01:44:30Z)
• Attentive Prototypes for Source-free Unsupervised Domain Adaptive 3D Object Detection [85.11649974840758]
  3D object detection networks tend to be biased towards the data they are trained on. We propose a single-frame approach for source-free, unsupervised domain adaptation of lidar-based 3D object detectors.
  arXiv  Detail & Related papers  (2021-11-30T18:42:42Z)
• Automated Antenna Testing Using Encoder-Decoder-based Anomaly Detection [2.580765958706854]
  We propose a new method for testing antenna arrays that records the radiating electromagnetic (EM) field using an absorbing material. We are able to reconstruct normal sequences through our trained model and compare it to the real sequences observed by a thermal camera. A contour-based anomaly detector can then map the reconstruction error matrix to an anomaly score to identify faulty antenna arrays.
  arXiv  Detail & Related papers  (2021-11-27T12:46:25Z)
• Hankel-structured Tensor Robust PCA for Multivariate Traffic Time Series Anomaly Detection [9.067182100565695]
  This study proposes a Hankel-structured tensor version of RPCA for anomaly detection in spatial data. We decompose the corrupted matrix into a low-rank Hankel tensor and a sparse matrix. We evaluate the method by synthetic data and passenger flow time series.
  arXiv  Detail & Related papers  (2021-10-08T19:35:39Z)
• FMNet: Latent Feature-wise Mapping Network for Cleaning up Noisy Micro-Doppler Spectrogram [2.9849405664643585]
  noisy surroundings cause uninterpretable motion patterns on the micro-Doppler spectrogram. radar returns often suffer from multipath, clutter and interference. We propose a latent feature-wise mapping strategy, called Feature Mapping Network (FMNet), to transform measured spectrograms.
  arXiv  Detail & Related papers  (2021-07-09T19:20:41Z)
• Dense Label Encoding for Boundary Discontinuity Free Rotation Detection [69.75559390700887]
  This paper explores a relatively less-studied methodology based on classification. We propose new techniques to push its frontier in two aspects. Experiments and visual analysis on large-scale public datasets for aerial images show the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-11-19T05:42:02Z)
• On the Arbitrary-Oriented Object Detection: Classification based Approaches Revisited [94.5455251250471]
  We first show that the boundary problem suffered in existing dominant regression-based rotation detectors, is caused by angular periodicity or corner ordering. We transform the angular prediction task from a regression problem to a classification one. For the resulting circularly distributed angle classification problem, we first devise a Circular Smooth Label technique to handle the periodicity of angle and increase the error tolerance to adjacent angles.
  arXiv  Detail & Related papers  (2020-03-12T03:23:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.