Self-Healing Machine Learning: A Framework for Autonomous Adaptation in Real-World Environments

• URL: http://arxiv.org/abs/2411.00186v1
• Date: Thu, 31 Oct 2024 20:05:51 GMT
• Title: Self-Healing Machine Learning: A Framework for Autonomous Adaptation in Real-World Environments
• Authors: Paulius Rauba, Nabeel Seedat, Krzysztof Kacprzyk, Mihaela van der Schaar,
• Abstract summary: Real-world machine learning systems often encounter model performance degradation due to distributional shifts in the underlying data generating process. Existing approaches to addressing shifts, such as concept drift adaptation, are limited by their reason-agnostic nature. We propose self-healing machine learning (SHML) to overcome these limitations.
• Score: 50.310636905746975
• License:
• Abstract: Real-world machine learning systems often encounter model performance degradation due to distributional shifts in the underlying data generating process (DGP). Existing approaches to addressing shifts, such as concept drift adaptation, are limited by their reason-agnostic nature. By choosing from a pre-defined set of actions, such methods implicitly assume that the causes of model degradation are irrelevant to what actions should be taken, limiting their ability to select appropriate adaptations. In this paper, we propose an alternative paradigm to overcome these limitations, called self-healing machine learning (SHML). Contrary to previous approaches, SHML autonomously diagnoses the reason for degradation and proposes diagnosis-based corrective actions. We formalize SHML as an optimization problem over a space of adaptation actions to minimize the expected risk under the shifted DGP. We introduce a theoretical framework for self-healing systems and build an agentic self-healing solution H-LLM which uses large language models to perform self-diagnosis by reasoning about the structure underlying the DGP, and self-adaptation by proposing and evaluating corrective actions. Empirically, we analyze different components of H-LLM to understand why and when it works, demonstrating the potential of self-healing ML.

Related papers

• Generative Principal Component Regression via Variational Inference [2.4415762506639944]
  One approach to designing appropriate manipulations is to target key features of predictive models. We develop a novel objective based on supervised variational autoencoders (SVAEs) that enforces such information is represented in the latent space. We show in simulations that gPCR dramatically improves target selection in manipulation as compared to standard PCR and SVAEs.
  arXiv  Detail & Related papers  (2024-09-03T22:38:55Z)
• Self-Labeling in Multivariate Causality and Quantification for Adaptive Machine Learning [0.0]
  An interactive causality based self-labeling method was proposed to autonomously associate causally related data streams for domain adaptation. This paper further develops the self-labeling framework and its theoretical foundations to address these research questions.
  arXiv  Detail & Related papers  (2024-04-08T18:16:22Z)
• Tuning-Free Accountable Intervention for LLM Deployment -- A Metacognitive Approach [55.613461060997004]
  Large Language Models (LLMs) have catalyzed transformative advances across a spectrum of natural language processing tasks. We propose an innovative textitmetacognitive approach, dubbed textbfCLEAR, to equip LLMs with capabilities for self-aware error identification and correction.
  arXiv  Detail & Related papers  (2024-03-08T19:18:53Z)
• InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance [56.184255657175335]
  We develop textbfInferAligner, a novel inference-time alignment method that utilizes cross-model guidance for harmlessness alignment. Experimental results show that our method can be very effectively applied to domain-specific models in finance, medicine, and mathematics. It significantly diminishes the Attack Success Rate (ASR) of both harmful instructions and jailbreak attacks, while maintaining almost unchanged performance in downstream tasks.
  arXiv  Detail & Related papers  (2024-01-20T10:41:03Z)
• Predictable MDP Abstraction for Unsupervised Model-Based RL [93.91375268580806]
  We propose predictable MDP abstraction (PMA) Instead of training a predictive model on the original MDP, we train a model on a transformed MDP with a learned action space. We theoretically analyze PMA and empirically demonstrate that PMA leads to significant improvements over prior unsupervised model-based RL approaches.
  arXiv  Detail & Related papers  (2023-02-08T07:37:51Z)
• Learning Optimal Features via Partial Invariance [18.552839725370383]
  Invariant Risk Minimization (IRM) is a popular framework that aims to learn robust models from multiple environments. We show that IRM can over-constrain the predictor and to remedy this, we propose a relaxation via $textitpartial invariance$. Several experiments, conducted both in linear settings as well as with deep neural networks on tasks over both language and image data, allow us to verify our conclusions.
  arXiv  Detail & Related papers  (2023-01-28T02:48:14Z)
• When to Update Your Model: Constrained Model-based Reinforcement Learning [50.74369835934703]
  We propose a novel and general theoretical scheme for a non-decreasing performance guarantee of model-based RL (MBRL) Our follow-up derived bounds reveal the relationship between model shifts and performance improvement. A further example demonstrates that learning models from a dynamically-varying number of explorations benefit the eventual returns.
  arXiv  Detail & Related papers  (2022-10-15T17:57:43Z)
• HEX: Human-in-the-loop Explainability via Deep Reinforcement Learning [2.322461721824713]
  We propose HEX, a human-in-the-loop deep reinforcement learning approach to machine learning explainability (MLX) Our formulation explicitly considers the decision boundary of the ML model in question, rather than the underlying training data. Our proposed methods thus synthesize HITL MLX policies that explicitly capture the decision boundary of the model in question for use in limited data scenarios.
  arXiv  Detail & Related papers  (2022-06-02T23:53:40Z)
• On the Impact of Applying Machine Learning in the Decision-Making of Self-Adaptive Systems [17.93069260609691]
  We use computational learning theory to determine a theoretical bound on the impact of the machine learning method on the predictions made by the verifier. To conclude, we look at opportunities for future research in this area.
  arXiv  Detail & Related papers  (2021-03-18T11:59:50Z)
• Transfer Learning without Knowing: Reprogramming Black-box Machine Learning Models with Scarce Data and Limited Resources [78.72922528736011]
  We propose a novel approach, black-box adversarial reprogramming (BAR), that repurposes a well-trained black-box machine learning model. Using zeroth order optimization and multi-label mapping techniques, BAR can reprogram a black-box ML model solely based on its input-output responses. BAR outperforms state-of-the-art methods and yields comparable performance to the vanilla adversarial reprogramming method.
  arXiv  Detail & Related papers  (2020-07-17T01:52:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.