On Evaluating Self-Adaptive and Self-Healing Systems using Chaos Engineering

• URL: http://arxiv.org/abs/2208.13227v1
• Date: Sun, 28 Aug 2022 14:38:57 GMT
• Title: On Evaluating Self-Adaptive and Self-Healing Systems using Chaos Engineering
• Abstract summary: We propose CHESS, an approach for the systematic evaluation of self-adaptive and self-healing systems. Chaos engineering is a methodology for subjecting a system to unexpected conditions and scenarios. We investigate the viability of this approach through an exploratory study on a self-healing smart office environment.
• Score: 0.6117371161379209
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the growing adoption of self-adaptive systems in various domains, there is an increasing need for strategies to assess their correct behavior. In particular self-healing systems, which aim to provide resilience and fault-tolerance, often deal with unanticipated failures in critical and highly dynamic environments. Their reactive and complex behavior makes it challenging to assess if these systems execute according to the desired goals. Recently, several studies have expressed concern about the lack of systematic evaluation methods for self-healing behavior. In this paper, we propose CHESS, an approach for the systematic evaluation of self-adaptive and self-healing systems that builds on chaos engineering. Chaos engineering is a methodology for subjecting a system to unexpected conditions and scenarios. It has shown great promise in helping developers build resilient microservice architectures and cyber-physical systems. CHESS turns this idea around by using chaos engineering to evaluate how well a self-healing system can withstand such perturbations. We investigate the viability of this approach through an exploratory study on a self-healing smart office environment. The study helps us explore the promises and limitations of the approach, as well as identify directions where additional work is needed. We conclude with a summary of lessons learned.

Related papers

• Why AI systems don't learn and what to do about it: Lessons on autonomous learning from cognitive science [79.50667364149979]
  We critically examine the limitations of current AI models in achieving autonomous learning and propose a learning architecture inspired by human and animal cognition.<n>The proposed framework integrates learning from observation (System A) and learning from active behavior (System B) while switching flexibly between these learning modes as a function of internally generated meta-control signals (System M).
  arXiv  Detail & Related papers  (2026-03-16T14:54:56Z)
• A Survey of Reasoning in Autonomous Driving Systems: Open Challenges and Emerging Paradigms [49.66022971508878]
  We argue that reasoning should be elevated from a modular component to the system's cognitive core.<n>We derive and systematize seven core reasoning challenges, such as the responsiveness-reasoning trade-off and social-game reasoning.<n>We identify a fundamental and unresolved tension between the high-latency, deliberative nature of LLM-based reasoning and the millisecond-scale, safety-critical demands of vehicle control.
  arXiv  Detail & Related papers  (2026-03-11T07:40:53Z)
• SoK: Agentic Retrieval-Augmented Generation (RAG): Taxonomy, Architectures, Evaluation, and Research Directions [0.0]
  This paper provides the first unified framework for understanding these autonomous systems.<n>We formalize agentic retrieval-generation loops as finite-horizon partially observable Markov decision processes.<n>We identify severe systemic risks inherent to autonomous loops, including compounding hallucination propagation, memory poisoning, retrieval misalignment, and cascading tool-execution vulnerabilities.
  arXiv  Detail & Related papers  (2026-03-07T23:38:57Z)
• Fundamentals of Building Autonomous LLM Agents [64.39018305018904]
  This paper reviews the architecture and implementation methods of agents powered by large language models (LLMs)<n>The research aims to explore patterns to develop "agentic" LLMs that can automate complex tasks and bridge the performance gap with human capabilities.
  arXiv  Detail & Related papers  (2025-10-10T10:32:39Z)
• Barbarians at the Gate: How AI is Upending Systems Research [58.95406995634148]
  We argue that systems research, long focused on designing and evaluating new performance-oriented algorithms, is particularly well-suited for AI-driven solution discovery.<n>We term this approach as AI-Driven Research for Systems ( ADRS), which iteratively generates, evaluates, and refines solutions.<n>Our results highlight both the disruptive potential and the urgent need to adapt systems research practices in the age of AI.
  arXiv  Detail & Related papers  (2025-10-07T17:49:24Z)
• A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems [53.37728204835912]
  Most existing AI systems rely on manually crafted configurations that remain static after deployment.<n>Recent research has explored agent evolution techniques that aim to automatically enhance agent systems based on interaction data and environmental feedback.<n>This survey aims to provide researchers and practitioners with a systematic understanding of self-evolving AI agents.
  arXiv  Detail & Related papers  (2025-08-10T16:07:32Z)
• Metacognition for Unknown Situations and Environments (MUSE) [3.2020845462590697]
  We propose the Metacognition for Unknown Situations and Environments (MUSE) framework. MUSE integrates metacognitive processes--specifically self-awareness and self-regulation--into autonomous agents. Agents show significant improvements in self-awareness and self-regulation.
  arXiv  Detail & Related papers  (2024-11-20T18:41:03Z)
• Self-Healing Machine Learning: A Framework for Autonomous Adaptation in Real-World Environments [50.310636905746975]
  Real-world machine learning systems often encounter model performance degradation due to distributional shifts in the underlying data generating process. Existing approaches to addressing shifts, such as concept drift adaptation, are limited by their reason-agnostic nature. We propose self-healing machine learning (SHML) to overcome these limitations.
  arXiv  Detail & Related papers  (2024-10-31T20:05:51Z)
• A Survey on Self-healing Software System [0.0]
  The main purpose of self-healing is to have an automatic system that can heal itself without human intervention. In this study, different self-healing methods are categorized and a summary of them is presented.
  arXiv  Detail & Related papers  (2024-03-01T11:23:41Z)
• Incremental procedural and sensorimotor learning in cognitive humanoid robots [52.77024349608834]
  This work presents a cognitive agent that can learn procedures incrementally. We show the cognitive functions required in each substage and how adding new functions helps address tasks previously unsolved by the agent. Results show that this approach is capable of solving complex tasks incrementally.
  arXiv  Detail & Related papers  (2023-04-30T22:51:31Z)
• Interactive System-wise Anomaly Detection [66.3766756452743]
  Anomaly detection plays a fundamental role in various applications. It is challenging for existing methods to handle the scenarios where the instances are systems whose characteristics are not readily observed as data. We develop an end-to-end approach which includes an encoder-decoder module that learns system embeddings.
  arXiv  Detail & Related papers  (2023-04-21T02:20:24Z)
• CHESS: A Framework for Evaluation of Self-adaptive Systems based on Chaos Engineering [0.6875312133832078]
  There is an increasing need to assess the correct behavior of self-adaptive and self-healing systems. There is a lack of systematic evaluation methods for self-adaptive and self-healing systems. We propose CHESS to address this gap by evaluating self-adaptive and self-healing systems through fault injection based on chaos engineering.
  arXiv  Detail & Related papers  (2023-03-13T17:00:55Z)
• A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems [128.63953314853327]
  "Lifelong Learning" systems are capable of 1) Continuous Learning, 2) Transfer and Adaptation, and 3) Scalability. We show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems.
  arXiv  Detail & Related papers  (2023-01-18T21:58:54Z)
• Multi Agent System for Machine Learning Under Uncertainty in Cyber Physical Manufacturing System [78.60415450507706]
  Recent advancements in predictive machine learning has led to its application in various use cases in manufacturing. Most research focused on maximising predictive accuracy without addressing the uncertainty associated with it. In this paper, we determine the sources of uncertainty in machine learning and establish the success criteria of a machine learning system to function well under uncertainty.
  arXiv  Detail & Related papers  (2021-07-28T10:28:05Z)
• Adaptive Autonomy in Human-on-the-Loop Vision-Based Robotics Systems [16.609594839630883]
  Computer vision approaches are widely used by autonomous robotic systems to guide their decision making. High accuracy is critical, particularly for Human-on-the-loop (HoTL) systems where humans play only a supervisory role. We propose a solution based upon adaptive autonomy levels, whereby the system detects loss of reliability of these models.
  arXiv  Detail & Related papers  (2021-03-28T05:43:10Z)
• Adaptive Immunity for Software: Towards Autonomous Self-healing Systems [0.6117371161379209]
  Self-healing software systems can detect, diagnose, and contain unanticipated problems at runtime. Recent advances in machine learning may be learned by observing the system. Artificial immune systems are particularly well-suited for building self-healing systems.
  arXiv  Detail & Related papers  (2021-01-07T13:22:55Z)
• Towards robust sensing for Autonomous Vehicles: An adversarial perspective [82.83630604517249]
  It is of primary importance that the resulting decisions are robust to perturbations. Adversarial perturbations are purposefully crafted alterations of the environment or of the sensory measurements. A careful evaluation of the vulnerabilities of their sensing system(s) is necessary in order to build and deploy safer systems.
  arXiv  Detail & Related papers  (2020-07-14T05:25:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.