The PetShop Dataset -- Finding Causes of Performance Issues across Microservices

• URL: http://arxiv.org/abs/2311.04806v2
• Date: Mon, 8 Apr 2024 18:34:42 GMT
• Title: The PetShop Dataset -- Finding Causes of Performance Issues across Microservices
• Authors: Michaela Hardt, William R. Orchard, Patrick Blöbaum, Shiva Kasiviswanathan, Elke Kirschbaum,
• Abstract summary: This paper introduces a dataset specifically designed for evaluating root cause analyses in microservice-based applications. The dataset encompasses latency, requests, and availability metrics emitted in 5-minute intervals from a distributed application. In addition to normal operation metrics, the dataset includes 68 injected performance issues, which increase latency and reduce availability throughout the system.
• Score: 3.87228935312714
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Identifying root causes for unexpected or undesirable behavior in complex systems is a prevalent challenge. This issue becomes especially crucial in modern cloud applications that employ numerous microservices. Although the machine learning and systems research communities have proposed various techniques to tackle this problem, there is currently a lack of standardized datasets for quantitative benchmarking. Consequently, research groups are compelled to create their own datasets for experimentation. This paper introduces a dataset specifically designed for evaluating root cause analyses in microservice-based applications. The dataset encompasses latency, requests, and availability metrics emitted in 5-minute intervals from a distributed application. In addition to normal operation metrics, the dataset includes 68 injected performance issues, which increase latency and reduce availability throughout the system. We showcase how this dataset can be used to evaluate the accuracy of a variety of methods spanning different causal and non-causal characterisations of the root cause analysis problem. We hope the new dataset, available at https://github.com/amazon-science/petshop-root-cause-analysis/ enables further development of techniques in this important area.

Related papers

• GAL-MAD: Towards Explainable Anomaly Detection in Microservice Applications Using Graph Attention Networks [1.0136215038345013]
  Anomalies stemming from network and performance issues must be swiftly identified and addressed. Existing anomaly detection techniques often rely on statistical models or machine learning methods. We propose a novel anomaly detection model called Graph Attention and LSTM-based Microservice Anomaly Detection (GAL-MAD)
  arXiv  Detail & Related papers  (2025-03-31T10:11:31Z)
• Continuous Observability Assurance in Cloud-Native Applications [0.0]
  We build on previous work and integrate our observability experiment tool OXN into a novel method for continuous observability assurance. We demonstrate its use and discuss future directions.
  arXiv  Detail & Related papers  (2025-03-11T15:43:26Z)
• A Comprehensive Library for Benchmarking Multi-class Visual Anomaly Detection [52.228708947607636]
  This paper introduces a comprehensive visual anomaly detection benchmark, ADer, which is a modular framework for new methods. The benchmark includes multiple datasets from industrial and medical domains, implementing fifteen state-of-the-art methods and nine comprehensive metrics. We objectively reveal the strengths and weaknesses of different methods and provide insights into the challenges and future directions of multi-class visual anomaly detection.
  arXiv  Detail & Related papers  (2024-06-05T13:40:07Z)
• Multi-modal Causal Structure Learning and Root Cause Analysis [67.67578590390907]
  We propose Mulan, a unified multi-modal causal structure learning method for root cause localization. We leverage a log-tailored language model to facilitate log representation learning, converting log sequences into time-series data. We also introduce a novel key performance indicator-aware attention mechanism for assessing modality reliability and co-learning a final causal graph.
  arXiv  Detail & Related papers  (2024-02-04T05:50:38Z)
• Root Cause Analysis In Microservice Using Neural Granger Causal Discovery [12.35924469567586]
  We propose RUN, a novel approach for root cause analysis using neural Granger causal discovery with contrastive learning. RUN enhances the backbone encoder by integrating contextual information from time series, and leverages a time series forecasting model to conduct neural Granger causal discovery. In addition, RUN incorporates Pagerank with a vector to efficiently recommend the top-k root causes.
  arXiv  Detail & Related papers  (2024-02-02T04:43:06Z)
• BESTMVQA: A Benchmark Evaluation System for Medical Visual Question Answering [8.547600133510551]
  This paper develops a Benchmark Evaluation SysTem for Medical Visual Question Answering, denoted by BESTMVQA. Our system provides a useful tool for users to automatically build Med-VQA datasets, which helps overcoming the data insufficient problem. With simple configurations, our system automatically trains and evaluates the selected models over a benchmark dataset.
  arXiv  Detail & Related papers  (2023-12-13T03:08:48Z)
• Zero-shot Retrieval: Augmenting Pre-trained Models with Search Engines [83.65380507372483]
  Large pre-trained models can dramatically reduce the amount of task-specific data required to solve a problem, but they often fail to capture domain-specific nuances out of the box. This paper shows how to leverage recent advances in NLP and multi-modal learning to augment a pre-trained model with search engine retrieval.
  arXiv  Detail & Related papers  (2023-11-29T05:33:28Z)
• Solving Data Quality Problems with Desbordante: a Demo [35.75243108496634]
  Desbordante is an open-source data profiler that aims to close this gap. It is built with emphasis on industrial application: it is efficient, scalable, resilient to crashes, and provides explanations. In this demonstration, we show several scenarios that allow end users to solve different data quality problems.
  arXiv  Detail & Related papers  (2023-07-27T15:26:26Z)
• LargeST: A Benchmark Dataset for Large-Scale Traffic Forecasting [65.71129509623587]
  Road traffic forecasting plays a critical role in smart city initiatives and has experienced significant advancements thanks to the power of deep learning. However, the promising results achieved on current public datasets may not be applicable to practical scenarios. We introduce the LargeST benchmark dataset, which includes a total of 8,600 sensors in California with a 5-year time coverage.
  arXiv  Detail & Related papers  (2023-06-14T05:48:36Z)
• Quality In / Quality Out: Assessing Data quality in an Anomaly Detection Benchmark [0.13764085113103217]
  We show that relatively minor modifications on the same benchmark dataset (UGR'16, a flow-based real-traffic dataset for anomaly detection) cause significantly more impact on model performance than the specific Machine Learning technique considered. Our findings illustrate the need to devote more attention into (automatic) data quality assessment and optimization techniques in the context of autonomous networks.
  arXiv  Detail & Related papers  (2023-05-31T12:03:12Z)
• Causal Scene BERT: Improving object detection by searching for challenging groups of data [125.40669814080047]
  Computer vision applications rely on learning-based perception modules parameterized with neural networks for tasks like object detection. These modules frequently have low expected error overall but high error on atypical groups of data due to biases inherent in the training process. Our main contribution is a pseudo-automatic method to discover such groups in foresight by performing causal interventions on simulated scenes.
  arXiv  Detail & Related papers  (2022-02-08T05:14:16Z)
• Complex Sequential Data Analysis: A Systematic Literature Review of Existing Algorithms [0.9649642656207869]
  This paper reviews past approaches to the use of deep-learning frameworks for the analysis of irregular-patterned datasets. Traditional deep-learning methods perform poorly or even fail when trying to analyse these datasets. The performance of deep-learning frameworks was found to be evaluated mainly using mean absolute error and root mean square error accuracy metrics.
  arXiv  Detail & Related papers  (2020-07-22T17:53:00Z)
• Data Mining with Big Data in Intrusion Detection Systems: A Systematic Literature Review [68.15472610671748]
  Cloud computing has become a powerful and indispensable technology for complex, high performance and scalable computation. The rapid rate and volume of data creation has begun to pose significant challenges for data management and security. The design and deployment of intrusion detection systems (IDS) in the big data setting has, therefore, become a topic of importance.
  arXiv  Detail & Related papers  (2020-05-23T20:57:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.