Learning Unstable Dynamical Systems with Time-Weighted Logarithmic Loss
- URL: http://arxiv.org/abs/2007.05189v1
- Date: Fri, 10 Jul 2020 06:28:05 GMT
- Title: Learning Unstable Dynamical Systems with Time-Weighted Logarithmic Loss
- Authors: Kamil Nar, Yuan Xue, Andrew M. Dai
- Abstract summary: We look into the dynamics of the gradient descent algorithm and pinpoint what causes the difficulty of learning unstable systems.
We introduce a time-weighted logarithmic loss function to fix this imbalance and demonstrate its effectiveness in learning unstable systems.
- Score: 20.167719985846002
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: When training the parameters of a linear dynamical model, the gradient
descent algorithm is likely to fail to converge if the squared-error loss is
used as the training loss function. Restricting the parameter space to a
smaller subset and running the gradient descent algorithm within this subset
can allow learning stable dynamical systems, but this strategy does not work
for unstable systems. In this work, we look into the dynamics of the gradient
descent algorithm and pinpoint what causes the difficulty of learning unstable
systems. We show that observations taken at different times from the system to
be learned influence the dynamics of the gradient descent algorithm in
substantially different degrees. We introduce a time-weighted logarithmic loss
function to fix this imbalance and demonstrate its effectiveness in learning
unstable systems.
Related papers
- Divide And Conquer: Learning Chaotic Dynamical Systems With Multistep Penalty Neural Ordinary Differential Equations [0.0]
Multistep Penalty NODE(MP-NODE) is applied to chaotic systems such as the Kuramoto-Sivashinsky equation and the two-dimensional Kolmogorov flow.
It can optimize chaotic systems in a manner similar to least-squares landscape with significantly lower computational costs.
arXiv Detail & Related papers (2024-06-30T02:50:28Z) - Differentially Flat Learning-based Model Predictive Control Using a
Stability, State, and Input Constraining Safety Filter [10.52705437098686]
Learning-based optimal control algorithms control unknown systems using past trajectory data and a learned model of the system dynamics.
We present a novel nonlinear controller that exploits differential flatness to achieve similar performance to state-of-the-art learning-based controllers.
arXiv Detail & Related papers (2023-07-20T02:42:23Z) - Losing momentum in continuous-time stochastic optimisation [62.997667081978825]
momentum-based algorithms have become especially popular in recent years.
In work, we propose and analyse a continuous-time model for gradient descent with momentum.
We show convergence of our system to the global minimiser when reducing momentum over time.
arXiv Detail & Related papers (2022-09-08T10:46:05Z) - A Priori Denoising Strategies for Sparse Identification of Nonlinear
Dynamical Systems: A Comparative Study [68.8204255655161]
We investigate and compare the performance of several local and global smoothing techniques to a priori denoise the state measurements.
We show that, in general, global methods, which use the entire measurement data set, outperform local methods, which employ a neighboring data subset around a local point.
arXiv Detail & Related papers (2022-01-29T23:31:25Z) - Bayesian Algorithms Learn to Stabilize Unknown Continuous-Time Systems [0.0]
Linear dynamical systems are canonical models for learning-based control of plants with uncertain dynamics.
A reliable stabilization procedure for this purpose that can effectively learn from unstable data to stabilize the system in a finite time is not currently available.
In this work, we propose a novel learning algorithm that stabilizes unknown continuous-time linear systems.
arXiv Detail & Related papers (2021-12-30T15:31:35Z) - Learning Unstable Dynamics with One Minute of Data: A
Differentiation-based Gaussian Process Approach [47.045588297201434]
We show how to exploit the differentiability of Gaussian processes to create a state-dependent linearized approximation of the true continuous dynamics.
We validate our approach by iteratively learning the system dynamics of an unstable system such as a 9-D segway.
arXiv Detail & Related papers (2021-03-08T05:08:47Z) - Gradient Starvation: A Learning Proclivity in Neural Networks [97.02382916372594]
Gradient Starvation arises when cross-entropy loss is minimized by capturing only a subset of features relevant for the task.
This work provides a theoretical explanation for the emergence of such feature imbalance in neural networks.
arXiv Detail & Related papers (2020-11-18T18:52:08Z) - Reinforcement Learning with Fast Stabilization in Linear Dynamical
Systems [91.43582419264763]
We study model-based reinforcement learning (RL) in unknown stabilizable linear dynamical systems.
We propose an algorithm that certifies fast stabilization of the underlying system by effectively exploring the environment.
We show that the proposed algorithm attains $tildemathcalO(sqrtT)$ regret after $T$ time steps of agent-environment interaction.
arXiv Detail & Related papers (2020-07-23T23:06:40Z) - Active Learning for Nonlinear System Identification with Guarantees [102.43355665393067]
We study a class of nonlinear dynamical systems whose state transitions depend linearly on a known feature embedding of state-action pairs.
We propose an active learning approach that achieves this by repeating three steps: trajectory planning, trajectory tracking, and re-estimation of the system from all available data.
We show that our method estimates nonlinear dynamical systems at a parametric rate, similar to the statistical rate of standard linear regression.
arXiv Detail & Related papers (2020-06-18T04:54:11Z) - Convergence and sample complexity of gradient methods for the model-free
linear quadratic regulator problem [27.09339991866556]
We show that ODE searches for optimal control for an unknown computation system by directly searching over the corresponding space of controllers.
We take a step towards demystifying the performance and efficiency of such methods by focusing on the gradient-flow dynamics set of stabilizing feedback gains and a similar result holds for the forward disctization of the ODE.
arXiv Detail & Related papers (2019-12-26T16:56:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.