Self-supervised Learning for Anomaly Detection in Computational Workflows

• URL: http://arxiv.org/abs/2310.01247v1
• Date: Mon, 2 Oct 2023 14:31:56 GMT
• Title: Self-supervised Learning for Anomaly Detection in Computational Workflows
• Authors: Hongwei Jin, Krishnan Raghavan, George Papadimitriou, Cong Wang, Anirban Mandal, Ewa Deelman, Prasanna Balaprakash
• Abstract summary: We introduce an autoencoder-driven self-supervised learning(SSL) approach that learns a summary statistic from unlabeled workflow data. In this approach, we combine generative and contrastive learning objectives to detect outliers in the summary statistics. We demonstrate that by estimating the distribution of normal behavior in the latent space, we can outperform state-of-the-art anomaly detection methods on our benchmark datasets.
• Score: 10.39119516144685
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Anomaly detection is the task of identifying abnormal behavior of a system. Anomaly detection in computational workflows is of special interest because of its wide implications in various domains such as cybersecurity, finance, and social networks. However, anomaly detection in computational workflows~(often modeled as graphs) is a relatively unexplored problem and poses distinct challenges. For instance, when anomaly detection is performed on graph data, the complex interdependency of nodes and edges, the heterogeneity of node attributes, and edge types must be accounted for. Although the use of graph neural networks can help capture complex inter-dependencies, the scarcity of labeled anomalous examples from workflow executions is still a significant challenge. To address this problem, we introduce an autoencoder-driven self-supervised learning~(SSL) approach that learns a summary statistic from unlabeled workflow data and estimates the normal behavior of the computational workflow in the latent space. In this approach, we combine generative and contrastive learning objectives to detect outliers in the summary statistics. We demonstrate that by estimating the distribution of normal behavior in the latent space, we can outperform state-of-the-art anomaly detection methods on our benchmark datasets.

Related papers

• Anomaly Detection by Context Contrasting [57.695202846009714]
  Anomaly detection focuses on identifying samples that deviate from the norm. Recent advances in self-supervised learning have shown great promise in this regard. We propose Con$$, which learns through context augmentations.
  arXiv  Detail & Related papers  (2024-05-29T07:59:06Z)
• Pattern-Based Time-Series Risk Scoring for Anomaly Detection and Alert Filtering -- A Predictive Maintenance Case Study [3.508168174653255]
  We propose a fast and efficient approach to anomaly detection and alert filtering based on sequential pattern similarities. We show how this approach can be leveraged for a variety of purposes involving anomaly detection on a large scale real-world industrial system.
  arXiv  Detail & Related papers  (2024-05-24T20:27:45Z)
• Detecting Anomalous Events in Object-centric Business Processes via Graph Neural Networks [55.583478485027]
  This study proposes a novel framework for anomaly detection in business processes. We first reconstruct the process dependencies of the object-centric event logs as attributed graphs. We then employ a graph convolutional autoencoder architecture to detect anomalous events.
  arXiv  Detail & Related papers  (2024-02-14T14:17:56Z)
• Multitask Active Learning for Graph Anomaly Detection [48.690169078479116]
  We propose a novel MultItask acTIve Graph Anomaly deTEction framework, namely MITIGATE. By coupling node classification tasks, MITIGATE obtains the capability to detect out-of-distribution nodes without known anomalies. Empirical studies on four datasets demonstrate that MITIGATE significantly outperforms the state-of-the-art methods for anomaly detection.
  arXiv  Detail & Related papers  (2024-01-24T03:43:45Z)
• WePaMaDM-Outlier Detection: Weighted Outlier Detection using Pattern Approaches for Mass Data Mining [0.6754597324022876]
  Outlier detection can reveal vital information about system faults, fraudulent activities, and patterns in the data. This article proposed the WePaMaDM-Outlier Detection with distinct mass data mining domain. It also investigates the significance of data modeling in outlier detection techniques in surveillance, fault detection, and trend analysis.
  arXiv  Detail & Related papers  (2023-06-09T07:00:00Z)
• Interactive System-wise Anomaly Detection [66.3766756452743]
  Anomaly detection plays a fundamental role in various applications. It is challenging for existing methods to handle the scenarios where the instances are systems whose characteristics are not readily observed as data. We develop an end-to-end approach which includes an encoder-decoder module that learns system embeddings.
  arXiv  Detail & Related papers  (2023-04-21T02:20:24Z)
• PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning [58.85063149619348]
  We propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows. Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets.
  arXiv  Detail & Related papers  (2023-01-25T16:34:43Z)
• Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning [50.24174211654775]
  We present a novel contrastive self-supervised learning framework for anomaly detection on attributed networks. Our framework fully exploits the local information from network data by sampling a novel type of contrastive instance pair. A graph neural network-based contrastive learning model is proposed to learn informative embedding from high-dimensional attributes and local structure.
  arXiv  Detail & Related papers  (2021-02-27T03:17:20Z)
• Video Anomaly Detection by Estimating Likelihood of Representations [21.879366166261228]
  Video anomaly is a challenging task because it involves solving many sub-tasks such as motion representation, object localization and action recognition. Traditionally, solutions to this task have focused on the mapping between video frames and their low-dimensional features, while ignoring the spatial connections of those features. Recent solutions focus on analyzing these spatial connections by using hard clustering techniques, such as K-Means, or applying neural networks to map latent features to a general understanding. In order to solve video anomaly in the latent feature space, we propose a deep probabilistic model to transfer this task into a density estimation problem.
  arXiv  Detail & Related papers  (2020-12-02T19:16:22Z)
• Anomaly Detection in Trajectory Data with Normalizing Flows [0.0]
  We propose an approach based on normalizing flows that enables complex density estimation from data with neural networks. Our proposal computes exact model likelihood values, an important feature of normalizing flows, for each segment of the trajectory. We evaluate our methodology, named aggregated anomaly detection with normalizing flows (GRADINGS), using real world trajectory data and compare it with more traditional anomaly detection techniques.
  arXiv  Detail & Related papers  (2020-04-13T14:16:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.