CHESS: A Framework for Evaluation of Self-adaptive Systems based on Chaos Engineering

• URL: http://arxiv.org/abs/2303.07283v1
• Date: Mon, 13 Mar 2023 17:00:55 GMT
• Title: CHESS: A Framework for Evaluation of Self-adaptive Systems based on Chaos Engineering
• Authors: Sehrish Malik, Moeen Ali Naqvi, Leon Moonen
• Abstract summary: There is an increasing need to assess the correct behavior of self-adaptive and self-healing systems. There is a lack of systematic evaluation methods for self-adaptive and self-healing systems. We propose CHESS to address this gap by evaluating self-adaptive and self-healing systems through fault injection based on chaos engineering.
• Score: 0.6875312133832078
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: There is an increasing need to assess the correct behavior of self-adaptive and self-healing systems due to their adoption in critical and highly dynamic environments. However, there is a lack of systematic evaluation methods for self-adaptive and self-healing systems. We proposed CHESS, a novel approach to address this gap by evaluating self-adaptive and self-healing systems through fault injection based on chaos engineering (CE) [ arXiv:2208.13227 ]. The artifact presented in this paper provides an extensive overview of the use of CHESS through two microservice-based case studies: a smart office case study and an existing demo application called Yelb. It comes with a managing system service, a self-monitoring service, as well as five fault injection scenarios covering infrastructure faults and functional faults. Each of these components can be easily extended or replaced to adopt the CHESS approach to a new case study, help explore its promises and limitations, and identify directions for future research. Keywords: self-healing, resilience, chaos engineering, evaluation, artifact

Related papers

• Self-Healing Machine Learning: A Framework for Autonomous Adaptation in Real-World Environments [50.310636905746975]
  Real-world machine learning systems often encounter model performance degradation due to distributional shifts in the underlying data generating process. Existing approaches to addressing shifts, such as concept drift adaptation, are limited by their reason-agnostic nature. We propose self-healing machine learning (SHML) to overcome these limitations.
  arXiv  Detail & Related papers  (2024-10-31T20:05:51Z)
• DARTH: Holistic Test-time Adaptation for Multiple Object Tracking [87.72019733473562]
  Multiple object tracking (MOT) is a fundamental component of perception systems for autonomous driving. Despite the urge of safety in driving systems, no solution to the MOT adaptation problem to domain shift in test-time conditions has ever been proposed. We introduce DARTH, a holistic test-time adaptation framework for MOT.
  arXiv  Detail & Related papers  (2023-10-03T10:10:42Z)
• Interactive System-wise Anomaly Detection [66.3766756452743]
  Anomaly detection plays a fundamental role in various applications. It is challenging for existing methods to handle the scenarios where the instances are systems whose characteristics are not readily observed as data. We develop an end-to-end approach which includes an encoder-decoder module that learns system embeddings.
  arXiv  Detail & Related papers  (2023-04-21T02:20:24Z)
• Awareness requirement and performance management for adaptive systems: a survey [13.406015141662879]
  Self-adaptive software can modify its behavior when the assessment indicates that the program is not performing as intended or when improved functionality or performance is available. This paper presents a review of self-adaptive systems in the context of requirement awareness and summarizes the most common methodologies applied.
  arXiv  Detail & Related papers  (2023-01-22T14:27:11Z)
• A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems [128.63953314853327]
  "Lifelong Learning" systems are capable of 1) Continuous Learning, 2) Transfer and Adaptation, and 3) Scalability. We show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems.
  arXiv  Detail & Related papers  (2023-01-18T21:58:54Z)
• On Evaluating Self-Adaptive and Self-Healing Systems using Chaos Engineering [0.6117371161379209]
  We propose CHESS, an approach for the systematic evaluation of self-adaptive and self-healing systems. Chaos engineering is a methodology for subjecting a system to unexpected conditions and scenarios. We investigate the viability of this approach through an exploratory study on a self-healing smart office environment.
  arXiv  Detail & Related papers  (2022-08-28T14:38:57Z)
• Learning Physical Concepts in Cyber-Physical Systems: A Case Study [72.74318982275052]
  We provide an overview of the current state of research regarding methods for learning physical concepts in time series data. We also analyze the most important methods from the current state of the art using the example of a three-tank system.
  arXiv  Detail & Related papers  (2021-11-28T14:24:52Z)
• On the Impact of Applying Machine Learning in the Decision-Making of Self-Adaptive Systems [17.93069260609691]
  We use computational learning theory to determine a theoretical bound on the impact of the machine learning method on the predictions made by the verifier. To conclude, we look at opportunities for future research in this area.
  arXiv  Detail & Related papers  (2021-03-18T11:59:50Z)
• Applying Machine Learning in Self-Adaptive Systems: A Systematic Literature Review [15.953995937484176]
  There is currently no systematic overview of the use of machine learning in self-adaptive systems. We focus on self-adaptive systems that are based on a traditional Monitor-Analyze-Plan-Execute feedback loop (MAPE) The research questions are centred on the problems that motivate the use of machine learning in self-adaptive systems, the key engineering aspects of learning in self-adaptation, and open challenges.
  arXiv  Detail & Related papers  (2021-03-06T13:45:59Z)
• Adaptive Immunity for Software: Towards Autonomous Self-healing Systems [0.6117371161379209]
  Self-healing software systems can detect, diagnose, and contain unanticipated problems at runtime. Recent advances in machine learning may be learned by observing the system. Artificial immune systems are particularly well-suited for building self-healing systems.
  arXiv  Detail & Related papers  (2021-01-07T13:22:55Z)
• Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts? [104.04999499189402]
  Out-of-training-distribution (OOD) scenarios are a common challenge of learning agents at deployment. We propose an uncertainty-aware planning method, called emphrobust imitative planning (RIP) Our method can detect and recover from some distribution shifts, reducing the overconfident and catastrophic extrapolations in OOD scenes. We introduce an autonomous car novel-scene benchmark, textttCARNOVEL, to evaluate the robustness of driving agents to a suite of tasks with distribution shifts.
  arXiv  Detail & Related papers  (2020-06-26T11:07:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.