Robust Failure Diagnosis of Microservice System through Multimodal Data

• URL: http://arxiv.org/abs/2302.10512v2
• Date: Wed, 31 May 2023 14:53:17 GMT
• Title: Robust Failure Diagnosis of Microservice System through Multimodal Data
• Authors: Shenglin Zhang, Pengxiang Jin, Zihan Lin, Yongqian Sun, Bicheng Zhang, Sibo Xia, Zhengdan Li, Zhenyu Zhong, Minghua Ma, Wa Jin, Dai Zhang, Zhenyu Zhu, Dan Pei
• Abstract summary: We propose DiagFusion, a robust failure diagnosis approach that uses multimodal data. Our evaluations show that DiagFusion outperforms existing methods in terms of root cause instance localization and failure type determination.
• Score: 14.720995687799668
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automatic failure diagnosis is crucial for large microservice systems. Currently, most failure diagnosis methods rely solely on single-modal data (i.e., using either metrics, logs, or traces). In this study, we conduct an empirical study using real-world failure cases to show that combining these sources of data (multimodal data) leads to a more accurate diagnosis. However, effectively representing these data and addressing imbalanced failures remain challenging. To tackle these issues, we propose DiagFusion, a robust failure diagnosis approach that uses multimodal data. It leverages embedding techniques and data augmentation to represent the multimodal data of service instances, combines deployment data and traces to build a dependency graph, and uses a graph neural network to localize the root cause instance and determine the failure type. Our evaluations using real-world datasets show that DiagFusion outperforms existing methods in terms of root cause instance localization (improving by 20.9% to 368%) and failure type determination (improving by 11.0% to 169%).

Related papers

• TVDiag: A Task-oriented and View-invariant Failure Diagnosis Framework with Multimodal Data [11.373761837547852]
  Microservice-based systems often suffer from reliability issues due to their intricate interactions and expanding scale. Traditional failure diagnosis methods that use single-modal data can hardly cover all failure scenarios due to the restricted information. We propose textitTVDiag, a multimodal failure diagnosis framework for locating culprit microservice instances and identifying their failure types.
  arXiv  Detail & Related papers  (2024-07-29T05:26:57Z)
• Multi-modal Causal Structure Learning and Root Cause Analysis [67.67578590390907]
  We propose Mulan, a unified multi-modal causal structure learning method for root cause localization. We leverage a log-tailored language model to facilitate log representation learning, converting log sequences into time-series data. We also introduce a novel key performance indicator-aware attention mechanism for assessing modality reliability and co-learning a final causal graph.
  arXiv  Detail & Related papers  (2024-02-04T05:50:38Z)
• Stochastic Amortization: A Unified Approach to Accelerate Feature and Data Attribution [62.71425232332837]
  We show that training amortized models with noisy labels is inexpensive and surprisingly effective. This approach significantly accelerates several feature attribution and data valuation methods, often yielding an order of magnitude speedup over existing approaches.
  arXiv  Detail & Related papers  (2024-01-29T03:42:37Z)
• Robust Multimodal Failure Detection for Microservice Systems [32.25907616511765]
  AnoFusion is an unsupervised failure detection approach for microservice systems. It learns the correlation of the heterogeneous multimodal data and integrates a Graph Attention Network (GAT) and Gated Recurrent Unit (GRU) It achieves the F1-score of 0.857 and 0.922, respectively, outperforming state-of-the-art failure detection approaches.
  arXiv  Detail & Related papers  (2023-05-30T12:39:42Z)
• Heterogeneous Anomaly Detection for Software Systems via Semi-supervised Cross-modal Attention [29.654681594903114]
  We propose Hades, the first end-to-end semi-supervised approach to identify system anomalies based on heterogeneous data. Our approach employs a hierarchical architecture to learn a global representation of the system status by fusing log semantics and metric patterns. We evaluate Hades extensively on large-scale simulated data and datasets from Huawei Cloud.
  arXiv  Detail & Related papers  (2023-02-14T09:02:11Z)
• Cluster-level pseudo-labelling for source-free cross-domain facial expression recognition [94.56304526014875]
  We propose the first Source-Free Unsupervised Domain Adaptation (SFUDA) method for Facial Expression Recognition (FER) Our method exploits self-supervised pretraining to learn good feature representations from the target data. We validate the effectiveness of our method in four adaptation setups, proving that it consistently outperforms existing SFUDA methods when applied to FER.
  arXiv  Detail & Related papers  (2022-10-11T08:24:50Z)
• On Modality Bias Recognition and Reduction [70.69194431713825]
  We study the modality bias problem in the context of multi-modal classification. We propose a plug-and-play loss function method, whereby the feature space for each label is adaptively learned. Our method yields remarkable performance improvements compared with the baselines.
  arXiv  Detail & Related papers  (2022-02-25T13:47:09Z)
• TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data [13.864161788250856]
  TranAD is a deep transformer network based anomaly detection and diagnosis model. It uses attention-based sequence encoders to swiftly perform inference with the knowledge of the broader temporal trends in the data. TranAD can outperform state-of-the-art baseline methods in detection and diagnosis performance with data and time-efficient training.
  arXiv  Detail & Related papers  (2022-01-18T19:41:29Z)
• A graph representation based on fluid diffusion model for multimodal data analysis: theoretical aspects and enhanced community detection [14.601444144225875]
  We introduce a novel model for graph definition based on fluid diffusion. Our method is able to strongly outperform state-of-the-art schemes for community detection in multimodal data analysis.
  arXiv  Detail & Related papers  (2021-12-07T16:30:03Z)
• A2Log: Attentive Augmented Log Anomaly Detection [53.06341151551106]
  Anomaly detection becomes increasingly important for the dependability and serviceability of IT services. Existing unsupervised methods need anomaly examples to obtain a suitable decision boundary. We develop A2Log, which is an unsupervised anomaly detection method consisting of two steps: Anomaly scoring and anomaly decision.
  arXiv  Detail & Related papers  (2021-09-20T13:40:21Z)
• TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks [73.01104041298031]
  TadGAN is an unsupervised anomaly detection approach built on Generative Adversarial Networks (GANs) To capture the temporal correlations of time series, we use LSTM Recurrent Neural Networks as base models for Generators and Critics. To demonstrate the performance and generalizability of our approach, we test several anomaly scoring techniques and report the best-suited one.
  arXiv  Detail & Related papers  (2020-09-16T15:52:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.