Stochastic Online Optimization for Cyber-Physical and Robotic Systems

• URL: http://arxiv.org/abs/2404.05318v1
• Date: Mon, 8 Apr 2024 09:08:59 GMT
• Title: Stochastic Online Optimization for Cyber-Physical and Robotic Systems
• Authors: Hao Ma, Melanie Zeilinger, Michael Muehlebach,
• Abstract summary: We propose a novel online framework for solving programming problems in the context of cyber-physical and robotic systems. Our problem formulation constraints model the evolution of a cyber-physical system, which has, in general, a continuous state and action space space is nonlinear. We show that even rough estimates of the dynamics can significantly improve the convergence of our algorithms.
• Score: 9.392372266209103
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel gradient-based online optimization framework for solving stochastic programming problems that frequently arise in the context of cyber-physical and robotic systems. Our problem formulation accommodates constraints that model the evolution of a cyber-physical system, which has, in general, a continuous state and action space, is nonlinear, and where the state is only partially observed. We also incorporate an approximate model of the dynamics as prior knowledge into the learning process and show that even rough estimates of the dynamics can significantly improve the convergence of our algorithms. Our online optimization framework encompasses both gradient descent and quasi-Newton methods, and we provide a unified convergence analysis of our algorithms in a non-convex setting. We also characterize the impact of modeling errors in the system dynamics on the convergence rate of the algorithms. Finally, we evaluate our algorithms in simulations of a flexible beam, a four-legged walking robot, and in real-world experiments with a ping-pong playing robot.

Related papers

• Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics [50.191655141020505]
  We introduce a novel framework for learning world models. By providing a scalable and robust framework, we pave the way for adaptive and efficient robotic systems in real-world applications.
  arXiv  Detail & Related papers  (2025-01-17T10:39:09Z)
• Model-Agnostic Zeroth-Order Policy Optimization for Meta-Learning of Ergodic Linear Quadratic Regulators [13.343937277604892]
  We study the problem of using meta-learning to deal with uncertainty and heterogeneity in ergodic linear quadratic regulators. We propose an algorithm that omits the estimation of policy Hessian, which applies to tasks of learning a set of heterogeneous but similar linear dynamic systems. We provide a convergence result for the exact gradient descent process by analyzing the boundedness and smoothness of the gradient for the meta-objective.
  arXiv  Detail & Related papers  (2024-05-27T17:26:36Z)
• Mechanistic Neural Networks for Scientific Machine Learning [58.99592521721158]
  We present Mechanistic Neural Networks, a neural network design for machine learning applications in the sciences. It incorporates a new Mechanistic Block in standard architectures to explicitly learn governing differential equations as representations. Central to our approach is a novel Relaxed Linear Programming solver (NeuRLP) inspired by a technique that reduces solving linear ODEs to solving linear programs.
  arXiv  Detail & Related papers  (2024-02-20T15:23:24Z)
• Adaptive learning of effective dynamics: Adaptive real-time, online modeling for complex systems [2.6144444305800234]
  We propose a novel framework that bridges large scale simulations and reduced order models to extract and forecast adaptively effective dynamics. AdaLED employs an autoencoder to identify reduced-order representations of the system dynamics and an ensemble of probabilistic recurrent neural networks (RNNs) as the latent time-steppertemporal. The framework alternates between the computational solver and the surrogate, accelerating learned dynamics while leaving yet-to-be-learned dynamics regimes to the original solver.
  arXiv  Detail & Related papers  (2023-04-04T12:05:51Z)
• Online Learning of Wheel Odometry Correction for Mobile Robots with Attention-based Neural Network [63.8376359764052]
  Modern robotic platforms need a reliable localization system to operate daily beside humans. Simple pose estimation algorithms based on filtered wheel and inertial odometry often fail in the presence of abrupt kinematic changes and wheel slips. We propose an innovative online learning approach for wheel odometry correction, paving the way for a robust multi-source localization system.
  arXiv  Detail & Related papers  (2023-03-21T10:30:31Z)
• Smoothed Online Learning for Prediction in Piecewise Affine Systems [43.64498536409903]
  This paper builds on the recently developed smoothed online learning framework. It provides the first algorithms for prediction and simulation in piecewise affine systems.
  arXiv  Detail & Related papers  (2023-01-26T15:54:14Z)
• Gradient-Based Trajectory Optimization With Learned Dynamics [80.41791191022139]
  We use machine learning techniques to learn a differentiable dynamics model of the system from data. We show that a neural network can model highly nonlinear behaviors accurately for large time horizons. In our hardware experiments, we demonstrate that our learned model can represent complex dynamics for both the Spot and Radio-controlled (RC) car.
  arXiv  Detail & Related papers  (2022-04-09T22:07:34Z)
• Constructing Neural Network-Based Models for Simulating Dynamical Systems [59.0861954179401]
  Data-driven modeling is an alternative paradigm that seeks to learn an approximation of the dynamics of a system using observations of the true system. This paper provides a survey of the different ways to construct models of dynamical systems using neural networks. In addition to the basic overview, we review the related literature and outline the most significant challenges from numerical simulations that this modeling paradigm must overcome.
  arXiv  Detail & Related papers  (2021-11-02T10:51:42Z)
• Multiscale Simulations of Complex Systems by Learning their Effective Dynamics [10.52078600986485]
  We present a systematic framework that bridges large scale simulations and reduced order models to Learn the Effective Dynamics. LED provides a novel potent modality for the accurate prediction of complex systems. LED is applicable to systems ranging from chemistry to fluid mechanics and reduces computational effort by up to two orders of magnitude.
  arXiv  Detail & Related papers  (2020-06-24T02:35:51Z)
• Information Theoretic Model Predictive Q-Learning [64.74041985237105]
  We present a novel theoretical connection between information theoretic MPC and entropy regularized RL. We develop a Q-learning algorithm that can leverage biased models.
  arXiv  Detail & Related papers  (2019-12-31T00:29:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.