Safe Sequential Optimization for Switching Environments

• URL: http://arxiv.org/abs/2311.02119v1
• Date: Fri, 3 Nov 2023 05:41:42 GMT
• Title: Safe Sequential Optimization for Switching Environments
• Abstract summary: We consider the problem of designing a sequential decision making agent to maximize an unknown time-varying function which switches with time. We propose a policy called Adaptive-SafeOpt and evaluate its performance via simulations.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We consider the problem of designing a sequential decision making agent to maximize an unknown time-varying function which switches with time. At each step, the agent receives an observation of the function's value at a point decided by the agent. The observation could be corrupted by noise. The agent is also constrained to take safe decisions with high probability, i.e., the chosen points should have a function value greater than a threshold. For this switching environment, we propose a policy called Adaptive-SafeOpt and evaluate its performance via simulations. The policy incorporates Bayesian optimization and change point detection for the safe sequential optimization problem. We observe that a major challenge in adapting to the switching change is to identify safe decisions when the change point is detected and prevent attraction to local optima.

Related papers

• Adaptive Agent Design [3.3263205689999453]
  We study the bi-level agent design problem that optimize the transition kernel and the policy it induces.<n>We show that a soft $Q$-learning algorithm converges almost surely to the fixed point of a soft Bellman equation.<n>In partially observed Markov decision problems, we analyze convergence properties of parametrized transition kernel design.
  arXiv  Detail & Related papers  (2026-09-11T06:32:40Z)
• Deciding When to Decide: Testing Operational Suboptimality Under Distributional Shift [6.370987174111389]
  We propose textttRADAR (Regret-based Assessment of Decision Adequacy and Risk), a decision-focused framework that uses inverse optimization to infer latent preferences.<n>By targeting regret, textttRADAR ignores decision-irrelevant shifts while detecting changes that warrant re-optimization.
  arXiv  Detail & Related papers  (2026-08-29T23:21:36Z)
• Conformal Policy Control [50.46542384484142]
  We show how to use any safe reference policy as a probabilistic regulator for any optimized but untested policy.<n>Unlike conservative optimization methods, we do not assume the user has identified the correct model class.<n>Our experiments on applications ranging from natural language question answering to biomolecular engineering show that safe exploration is possible from the first moment of deployment.
  arXiv  Detail & Related papers  (2026-03-02T18:54:36Z)
• Towards safe control parameter tuning in distributed multi-agent systems [10.487548576958421]
  Many safety-critical real-world problems, such as autonomous and collaborative robots, are of a distributed multi-agent nature.<n>To optimize the performance of these systems while ensuring safety, we can cast them as problems where each agent's parameters coupled to reward function coupled constraints.
  arXiv  Detail & Related papers  (2025-08-19T08:13:53Z)
• Rethinking Safety in LLM Fine-tuning: An Optimization Perspective [56.31306558218838]
  We show that poor optimization choices, rather than inherent trade-offs, often cause safety problems, measured as harmful responses to adversarial prompts.<n>We propose a simple exponential moving average (EMA) momentum technique in parameter space that preserves safety performance.<n>Our experiments on the Llama families across multiple datasets demonstrate that safety problems can largely be avoided without specialized interventions.
  arXiv  Detail & Related papers  (2025-08-17T23:46:36Z)
• Continual Adaptation: Environment-Conditional Parameter Generation for Object Detection in Dynamic Scenarios [54.58186816693791]
  environments constantly change over time and space, posing significant challenges for object detectors trained based on a closed-set assumption.<n>We propose a new mechanism, converting the fine-tuning process to a specific- parameter generation.<n>In particular, we first design a dual-path LoRA-based domain-aware adapter that disentangles features into domain-invariant and domain-specific components.
  arXiv  Detail & Related papers  (2025-06-30T17:14:12Z)
• Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environments [78.15330971155778]
  Posterior and Diversity Synergized Task Sampling (PDTS) is an easy-to-implement method to accommodate fast and robust sequential decision-making. PDTS unlocks the potential of robust active task sampling, significantly improves the zero-shot and few-shot adaptation robustness in challenging tasks, and even accelerates the learning process under certain scenarios.
  arXiv  Detail & Related papers  (2025-04-27T07:27:17Z)
• Adaptive Resampling with Bootstrap for Noisy Multi-Objective Optimization Problems [0.0]
  This paper presents a resampling decision function that incorporates the nature of the optimization problem by using bootstrapping and the probability of dominance. The efficiency of this resampling approach is demonstrated by applying it in the NSGA-II algorithm with a sequential resampling procedure under multiple noise variations.
  arXiv  Detail & Related papers  (2025-03-27T13:32:42Z)
• Safe Time-Varying Optimization based on Gaussian Processes with Spatio-Temporal Kernel [4.586346034304039]
  TVSafeOpt is an algorithm for time-varying optimization problems with unknown reward and safety functions. TVSafeOpt is capable of safely tracking a time-varying safe region without need for explicit change detection. We show that TVSafeOpt compares favorably against SafeOpt on synthetic data, both regarding safety and optimality.
  arXiv  Detail & Related papers  (2024-09-26T16:09:19Z)
• CMA-ES for Safe Optimization [9.130749109828717]
  This study focuses on CMA-ES as an efficient evolutionary algorithm and proposes an optimization method termed safe CMA-ES. The safe CMA-ES is designed to achieve both safety and efficiency in safe optimization.
  arXiv  Detail & Related papers  (2024-05-17T04:24:56Z)
• Information-Theoretic Safe Bayesian Optimization [59.758009422067005]
  We consider a sequential decision making task, where the goal is to optimize an unknown function without evaluating parameters that violate an unknown (safety) constraint. Most current methods rely on a discretization of the domain and cannot be directly extended to the continuous case. We propose an information-theoretic safe exploration criterion that directly exploits the GP posterior to identify the most informative safe parameters to evaluate.
  arXiv  Detail & Related papers  (2024-02-23T14:31:10Z)
• Adaptive Real Time Exploration and Optimization for Safety-Critical Systems [0.0]
  We propose the ARTEO algorithm, where we cast multi-armed bandits as a programming problem subject to safety constraints. We learn the environmental characteristics through changes in optimization inputs and through exploration. Compared to existing safe-learning approaches, our algorithm does not require an exclusive exploration phase and follows the optimization goals even in the explored points.
  arXiv  Detail & Related papers  (2022-11-10T11:37:22Z)
• Generalizing Bayesian Optimization with Decision-theoretic Entropies [102.82152945324381]
  We consider a generalization of Shannon entropy from work in statistical decision theory. We first show that special cases of this entropy lead to popular acquisition functions used in BO procedures. We then show how alternative choices for the loss yield a flexible family of acquisition functions.
  arXiv  Detail & Related papers  (2022-10-04T04:43:58Z)
• Meta-Learning Priors for Safe Bayesian Optimization [72.8349503901712]
  We build on a meta-learning algorithm, F-PACOH, capable of providing reliable uncertainty quantification in settings of data scarcity. As core contribution, we develop a novel framework for choosing safety-compliant priors in a data-riven manner. On benchmark functions and a high-precision motion system, we demonstrate that our meta-learned priors accelerate the convergence of safe BO approaches.
  arXiv  Detail & Related papers  (2022-10-03T08:38:38Z)
• Log Barriers for Safe Black-box Optimization with Application to Safe Reinforcement Learning [72.97229770329214]
  We introduce a general approach for seeking high dimensional non-linear optimization problems in which maintaining safety during learning is crucial. Our approach called LBSGD is based on applying a logarithmic barrier approximation with a carefully chosen step size. We demonstrate the effectiveness of our approach on minimizing violation in policy tasks in safe reinforcement learning.
  arXiv  Detail & Related papers  (2022-07-21T11:14:47Z)
• Off-Policy Evaluation with Policy-Dependent Optimization Response [90.28758112893054]
  We develop a new framework for off-policy evaluation with a textitpolicy-dependent linear optimization response. We construct unbiased estimators for the policy-dependent estimand by a perturbation method. We provide a general algorithm for optimizing causal interventions.
  arXiv  Detail & Related papers  (2022-02-25T20:25:37Z)
• Bayesian Persuasion for Algorithmic Recourse [28.586165301962485]
  In some situations, the underlying predictive model is deliberately kept secret to avoid gaming. This opacity forces the decision subjects to rely on incomplete information when making strategic feature modifications. We capture such settings as a game of Bayesian persuasion, in which the decision-maker sends a signal, e.g., an action recommendation, to a decision subject to incentivize them to take desirable actions.
  arXiv  Detail & Related papers  (2021-12-12T17:18:54Z)
• Optimal Sequential Detection of Signals with Unknown Appearance and Disappearance Points in Time [64.26593350748401]
  The paper addresses a sequential changepoint detection problem, assuming that the duration of change may be finite and unknown. We focus on a reliable maximin change detection criterion of maximizing the minimal probability of detection in a given time (or space) window. The FMA algorithm is applied to detecting faint streaks of satellites in optical images.
  arXiv  Detail & Related papers  (2021-02-02T04:58:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.