Online Learning for Supervisory Switching Control

• URL: http://arxiv.org/abs/2603.14762v2
• Date: Tue, 17 Mar 2026 21:11:18 GMT
• Title: Online Learning for Supervisory Switching Control
• Abstract summary: We study supervisory switching control for partially-observed linear dynamical systems.<n>The objective is to identify and deploy the best controller for the unknown system by selecting among a collection of $N$ candidate controllers.<n>We propose a novel, non-asymptotic analysis of supervisory control that adapts multi-armed bandit algorithms to a control-theoretic setting.
• Score: 32.18235763142726
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study supervisory switching control for partially-observed linear dynamical systems. The objective is to identify and deploy the best controller for the unknown system by periodically selecting among a collection of $N$ candidate controllers, some of which may destabilize the underlying system. While classical estimator-based supervisory control guarantees asymptotic stability, it lacks quantitative finite-time performance bounds. Conversely, current non-asymptotic methods in both online learning and system identification require restrictive assumptions that are incompatible in a control setting, such as system stability, which preclude testing potentially unstable controllers. To bridge this gap, we propose a novel, non-asymptotic analysis of supervisory control that adapts multi-armed bandit algorithms to a control-theoretic setting. The proposed data-driven algorithm evaluates candidate controllers via scoring criteria that leverage system observability to isolate the effects of state history, enabling both detection of destabilizing controllers and accurate system identification. We present two algorithmic variants with dimension-free, finite-time guarantees, where each identifies the most suitable controller in $\mathcal{O}(N \log N)$ steps, while simultaneously achieving finite $L_2$-gain with respect to system disturbances.

Related papers

• Neural Network-assisted Interval Reachability for Systems with Control Barrier Function-Based Safe Controllers [1.77513002450736]
  Control Barrier Functions (CBFs) have been widely utilized in the design of optimization-based controllers and filters for dynamical systems.<n>CBF-based controllers offer safety guarantees, but they can compromise the performance of the system.<n>We propose a computationally efficient interval reachimatability method for performance verification of systems with optimization-based controllers.
  arXiv  Detail & Related papers  (2025-04-11T04:14:55Z)
• Neural Port-Hamiltonian Models for Nonlinear Distributed Control: An Unconstrained Parametrization Approach [0.0]
  Neural Networks (NNs) can be leveraged to parametrize control policies that yield good performance. NNs' sensitivity to small input changes poses a risk of destabilizing the closed-loop system. To address these problems, we leverage the framework of port-Hamiltonian systems to design continuous-time distributed control policies. The effectiveness of the proposed distributed controllers is demonstrated through consensus control of non-holonomic mobile robots.
  arXiv  Detail & Related papers  (2024-11-15T10:44:29Z)
• Stability Bounds for Learning-Based Adaptive Control of Discrete-Time Multi-Dimensional Stochastic Linear Systems with Input Constraints [3.8004168340068336]
  We consider the problem of adaptive stabilization for discrete-time, multi-dimensional systems with bounded control input constraints and unbounded disturbances. We propose a certainty-equivalent control scheme which combines online parameter estimation with saturated linear control.
  arXiv  Detail & Related papers  (2023-04-02T16:38:13Z)
• Actor-Critic based Improper Reinforcement Learning [61.430513757337486]
  We consider an improper reinforcement learning setting where a learner is given $M$ base controllers for an unknown Markov decision process. We propose two algorithms: (1) a Policy Gradient-based approach; and (2) an algorithm that can switch between a simple Actor-Critic scheme and a Natural Actor-Critic scheme.
  arXiv  Detail & Related papers  (2022-07-19T05:55:02Z)
• Sparsity in Partially Controllable Linear Systems [56.142264865866636]
  We study partially controllable linear dynamical systems specified by an underlying sparsity pattern. Our results characterize those state variables which are irrelevant for optimal control.
  arXiv  Detail & Related papers  (2021-10-12T16:41:47Z)
• Stable Online Control of Linear Time-Varying Systems [49.41696101740271]
  COCO-LQ is an efficient online control algorithm that guarantees input-to-state stability for a large class of LTV systems. We empirically demonstrate the performance of COCO-LQ in both synthetic experiments and a power system frequency control example.
  arXiv  Detail & Related papers  (2021-04-29T06:18:49Z)
• A Meta-Learning Control Algorithm with Provable Finite-Time Guarantees [0.0]
  We provide provable regret guarantees for an online meta-learning control algorithm in an iterative control setting. We show that the worst regret for the learning within an iteration continuously improves with experience of more iterations.
  arXiv  Detail & Related papers  (2020-08-30T20:30:40Z)
• Learning Stabilizing Controllers for Unstable Linear Quadratic Regulators from a Single Trajectory [85.29718245299341]
  We study linear controllers under quadratic costs model also known as linear quadratic regulators (LQR) We present two different semi-definite programs (SDP) which results in a controller that stabilizes all systems within an ellipsoid uncertainty set. We propose an efficient data dependent algorithm -- textsceXploration -- that with high probability quickly identifies a stabilizing controller.
  arXiv  Detail & Related papers  (2020-06-19T08:58:57Z)
• Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting [91.43582419264763]
  We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
  arXiv  Detail & Related papers  (2020-03-12T19:56:38Z)
• Improper Learning for Non-Stochastic Control [78.65807250350755]
  We consider the problem of controlling a possibly unknown linear dynamical system with adversarial perturbations, adversarially chosen convex loss functions, and partially observed states. Applying online descent to this parametrization yields a new controller which attains sublinear regret vs. a large class of closed-loop policies. Our bounds are the first in the non-stochastic control setting that compete with emphall stabilizing linear dynamical controllers.
  arXiv  Detail & Related papers  (2020-01-25T02:12:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.