Learning Unstable Dynamical Systems with Time-Weighted Logarithmic Loss

• URL: http://arxiv.org/abs/2007.05189v1
• Date: Fri, 10 Jul 2020 06:28:05 GMT
• Title: Learning Unstable Dynamical Systems with Time-Weighted Logarithmic Loss
• Authors: Kamil Nar, Yuan Xue, Andrew M. Dai
• Abstract summary: We look into the dynamics of the gradient descent algorithm and pinpoint what causes the difficulty of learning unstable systems. We introduce a time-weighted logarithmic loss function to fix this imbalance and demonstrate its effectiveness in learning unstable systems.
• Score: 20.167719985846002
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: When training the parameters of a linear dynamical model, the gradient descent algorithm is likely to fail to converge if the squared-error loss is used as the training loss function. Restricting the parameter space to a smaller subset and running the gradient descent algorithm within this subset can allow learning stable dynamical systems, but this strategy does not work for unstable systems. In this work, we look into the dynamics of the gradient descent algorithm and pinpoint what causes the difficulty of learning unstable systems. We show that observations taken at different times from the system to be learned influence the dynamics of the gradient descent algorithm in substantially different degrees. We introduce a time-weighted logarithmic loss function to fix this imbalance and demonstrate its effectiveness in learning unstable systems.

Related papers

• Learning Unstable Continuous-Time Stochastic Linear Control Systems [0.0]
  We study the problem of system identification for continuous-time dynamics, based on a single finite-length state trajectory. We present a method for estimating the possibly unstable open-loop matrix by employing properly randomized control inputs. We establish theoretical performance guarantees showing that the estimation error decays with trajectory length, a measure of excitability, and the signal-to-noise ratio.
  arXiv  Detail & Related papers  (2024-09-17T16:24:51Z)
• Stochastic Reinforcement Learning with Stability Guarantees for Control of Unknown Nonlinear Systems [6.571209126567701]
  We propose a reinforcement learning algorithm that stabilizes the system by learning a local linear representation ofthe dynamics. We demonstrate the effectiveness of our algorithm on several challenging high-dimensional dynamical systems.
  arXiv  Detail & Related papers  (2024-09-12T20:07:54Z)
• Divide And Conquer: Learning Chaotic Dynamical Systems With Multistep Penalty Neural Ordinary Differential Equations [0.0]
  Multistep Penalty NODE is applied to chaotic systems such as the Kuramoto-Sivash Kolinsky equation, the two-dimensional Kolmogorov flow, and ERA5 reanalysis data for the atmosphere. It is observed that MPODE provide viable performance for such chaotic systems with significantly lower computational costs.
  arXiv  Detail & Related papers  (2024-06-30T02:50:28Z)
• A Priori Denoising Strategies for Sparse Identification of Nonlinear Dynamical Systems: A Comparative Study [68.8204255655161]
  We investigate and compare the performance of several local and global smoothing techniques to a priori denoise the state measurements. We show that, in general, global methods, which use the entire measurement data set, outperform local methods, which employ a neighboring data subset around a local point.
  arXiv  Detail & Related papers  (2022-01-29T23:31:25Z)
• Bayesian Algorithms Learn to Stabilize Unknown Continuous-Time Systems [0.0]
  Linear dynamical systems are canonical models for learning-based control of plants with uncertain dynamics. A reliable stabilization procedure for this purpose that can effectively learn from unstable data to stabilize the system in a finite time is not currently available. In this work, we propose a novel learning algorithm that stabilizes unknown continuous-time linear systems.
  arXiv  Detail & Related papers  (2021-12-30T15:31:35Z)
• Learning Unstable Dynamics with One Minute of Data: A Differentiation-based Gaussian Process Approach [47.045588297201434]
  We show how to exploit the differentiability of Gaussian processes to create a state-dependent linearized approximation of the true continuous dynamics. We validate our approach by iteratively learning the system dynamics of an unstable system such as a 9-D segway.
  arXiv  Detail & Related papers  (2021-03-08T05:08:47Z)
• Gradient Starvation: A Learning Proclivity in Neural Networks [97.02382916372594]
  Gradient Starvation arises when cross-entropy loss is minimized by capturing only a subset of features relevant for the task. This work provides a theoretical explanation for the emergence of such feature imbalance in neural networks.
  arXiv  Detail & Related papers  (2020-11-18T18:52:08Z)
• Training Generative Adversarial Networks by Solving Ordinary Differential Equations [54.23691425062034]
  We study the continuous-time dynamics induced by GAN training. From this perspective, we hypothesise that instabilities in training GANs arise from the integration error. We experimentally verify that well-known ODE solvers (such as Runge-Kutta) can stabilise training.
  arXiv  Detail & Related papers  (2020-10-28T15:23:49Z)
• Reinforcement Learning with Fast Stabilization in Linear Dynamical Systems [91.43582419264763]
  We study model-based reinforcement learning (RL) in unknown stabilizable linear dynamical systems. We propose an algorithm that certifies fast stabilization of the underlying system by effectively exploring the environment. We show that the proposed algorithm attains $tildemathcalO(sqrtT)$ regret after $T$ time steps of agent-environment interaction.
  arXiv  Detail & Related papers  (2020-07-23T23:06:40Z)
• Active Learning for Nonlinear System Identification with Guarantees [102.43355665393067]
  We study a class of nonlinear dynamical systems whose state transitions depend linearly on a known feature embedding of state-action pairs. We propose an active learning approach that achieves this by repeating three steps: trajectory planning, trajectory tracking, and re-estimation of the system from all available data. We show that our method estimates nonlinear dynamical systems at a parametric rate, similar to the statistical rate of standard linear regression.
  arXiv  Detail & Related papers  (2020-06-18T04:54:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.