Twin Graph-based Anomaly Detection via Attentive Multi-Modal Learning for Microservice System

• URL: http://arxiv.org/abs/2310.04701v1
• Date: Sat, 7 Oct 2023 06:28:41 GMT
• Title: Twin Graph-based Anomaly Detection via Attentive Multi-Modal Learning for Microservice System
• Authors: Jun Huang, Yang Yang, Hang Yu, Jianguo Li, Xiao Zheng
• Abstract summary: We propose MSTGAD, which seamlessly integrates all available data modalities via attentive multi-modal learning. We construct a transformer-based neural network with both spatial and temporal attention mechanisms to model the inter-correlations between different modalities. This enables us to detect anomalies automatically and accurately in real-time.
• Score: 24.2074235652359
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Microservice architecture has sprung up over recent years for managing enterprise applications, due to its ability to independently deploy and scale services. Despite its benefits, ensuring the reliability and safety of a microservice system remains highly challenging. Existing anomaly detection algorithms based on a single data modality (i.e., metrics, logs, or traces) fail to fully account for the complex correlations and interactions between different modalities, leading to false negatives and false alarms, whereas incorporating more data modalities can offer opportunities for further performance gain. As a fresh attempt, we propose in this paper a semi-supervised graph-based anomaly detection method, MSTGAD, which seamlessly integrates all available data modalities via attentive multi-modal learning. First, we extract and normalize features from the three modalities, and further integrate them using a graph, namely MST (microservice system twin) graph, where each node represents a service instance and the edge indicates the scheduling relationship between different service instances. The MST graph provides a virtual representation of the status and scheduling relationships among service instances of a real-world microservice system. Second, we construct a transformer-based neural network with both spatial and temporal attention mechanisms to model the inter-correlations between different modalities and temporal dependencies between the data points. This enables us to detect anomalies automatically and accurately in real-time. The source code of MSTGAD is publicly available at https://github.com/alipay/microservice_system_twin_graph_based_anomaly_detection.

Related papers

• System States Forecasting of Microservices with Dynamic Spatio-Temporal Data [9.519440926598524]
  Current forecasting methods are insufficient in environments where relationships are critical. In both short-term and long-term forecasting tasks, our model consistently achieved a 8.6% reduction in MAE(Mean Absolute Error) and a 2.2% reduction in MSE (Mean Squared Error)
  arXiv  Detail & Related papers  (2024-08-15T02:52:02Z)
• CHASE: A Causal Heterogeneous Graph based Framework for Root Cause Analysis in Multimodal Microservice Systems [22.00860661894853]
  We propose a Causal Heterogeneous grAph baSed framEwork for root cause analysis, namely CHASE, for microservice systems with multimodal data. CHASE learns from the constructed hypergraph with hyperedges representing the flow of causality and performs root cause localization.
  arXiv  Detail & Related papers  (2024-06-28T07:46:51Z)
• PeFAD: A Parameter-Efficient Federated Framework for Time Series Anomaly Detection [51.20479454379662]
  We propose a. Federated Anomaly Detection framework named PeFAD with the increasing privacy concerns. We conduct extensive evaluations on four real datasets, where PeFAD outperforms existing state-of-the-art baselines by up to 28.74%.
  arXiv  Detail & Related papers  (2024-06-04T13:51:08Z)
• Multitask Active Learning for Graph Anomaly Detection [48.690169078479116]
  We propose a novel MultItask acTIve Graph Anomaly deTEction framework, namely MITIGATE. By coupling node classification tasks, MITIGATE obtains the capability to detect out-of-distribution nodes without known anomalies. Empirical studies on four datasets demonstrate that MITIGATE significantly outperforms the state-of-the-art methods for anomaly detection.
  arXiv  Detail & Related papers  (2024-01-24T03:43:45Z)
• A Microservices Identification Method Based on Spectral Clustering for Industrial Legacy Systems [5.255685751491305]
  We propose an automated microservice decomposition method for extracting microservice candidates based on spectral graph theory. We show that our method can yield favorable results even without the involvement of domain experts.
  arXiv  Detail & Related papers  (2023-12-20T07:47:01Z)
• Fully-Connected Spatial-Temporal Graph for Multivariate Time-Series Data [50.84488941336865]
  We propose a novel method called Fully- Spatial-Temporal Graph Neural Network (FC-STGNN) For graph construction, we design a decay graph to connect sensors across all timestamps based on their temporal distances. For graph convolution, we devise FC graph convolution with a moving-pooling GNN layer to effectively capture the ST dependencies for learning effective representations.
  arXiv  Detail & Related papers  (2023-09-11T08:44:07Z)
• Robust Multimodal Failure Detection for Microservice Systems [32.25907616511765]
  AnoFusion is an unsupervised failure detection approach for microservice systems. It learns the correlation of the heterogeneous multimodal data and integrates a Graph Attention Network (GAT) and Gated Recurrent Unit (GRU) It achieves the F1-score of 0.857 and 0.922, respectively, outperforming state-of-the-art failure detection approaches.
  arXiv  Detail & Related papers  (2023-05-30T12:39:42Z)
• Efficient pattern-based anomaly detection in a network of multivariate devices [0.17188280334580192]
  We propose a scalable approach to detect anomalies using a two-step approach. First, we recover relations between entities in the network, since relations are often dynamic in nature and caused by an unknown underlying process. Next, we report anomalies based on an embedding of sequential patterns.
  arXiv  Detail & Related papers  (2023-05-07T16:05:30Z)
• Federated Stochastic Gradient Descent Begets Self-Induced Momentum [151.4322255230084]
  Federated learning (FL) is an emerging machine learning method that can be applied in mobile edge systems. We show that running to the gradient descent (SGD) in such a setting can be viewed as adding a momentum-like term to the global aggregation process.
  arXiv  Detail & Related papers  (2022-02-17T02:01:37Z)
• TELESTO: A Graph Neural Network Model for Anomaly Classification in Cloud Services [77.454688257702]
  Machine learning (ML) and artificial intelligence (AI) are applied on IT system operation and maintenance. One direction aims at the recognition of re-occurring anomaly types to enable remediation automation. We propose a method that is invariant to dimensionality changes of given data.
  arXiv  Detail & Related papers  (2021-02-25T14:24:49Z)
• Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks [91.65637773358347]
  We propose a general graph neural network framework designed specifically for multivariate time series data. Our approach automatically extracts the uni-directed relations among variables through a graph learning module. Our proposed model outperforms the state-of-the-art baseline methods on 3 of 4 benchmark datasets.
  arXiv  Detail & Related papers  (2020-05-24T04:02:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.