Reinforcement learning of rare diffusive dynamics

• URL: http://arxiv.org/abs/2105.04321v1
• Date: Mon, 10 May 2021 13:00:15 GMT
• Title: Reinforcement learning of rare diffusive dynamics
• Authors: Avishek Das, Dominic C. Rose, Juan P. Garrahan, David T. Limmer
• Abstract summary: We present a method to probe rare molecular dynamics trajectories directly using reinforcement learning. We consider trajectories conditioned to transition between regions of configuration space in finite time, as well as trajectories exhibiting rare fluctuations of time-integrated quantities in the long time limit. In both cases, reinforcement learning techniques are used to optimize an added force that minimizes the Kullback-Leibler divergence between the conditioned trajectory ensemble and a driven one.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a method to probe rare molecular dynamics trajectories directly using reinforcement learning. We consider trajectories that are conditioned to transition between regions of configuration space in finite time, like those relevant in the study of reactive events, as well as trajectories exhibiting rare fluctuations of time-integrated quantities in the long time limit, like those relevant in the calculation of large deviation functions. In both cases, reinforcement learning techniques are used to optimize an added force that minimizes the Kullback-Leibler divergence between the conditioned trajectory ensemble and a driven one. Under the optimized added force, the system evolves the rare fluctuation as a typical one, affording a variational estimate of its likelihood in the original trajectory ensemble. Low variance gradients employing value functions are proposed to increase the convergence of the optimal force. The method we develop employing these gradients leads to efficient and accurate estimates of both the optimal force and the likelihood of the rare event for a variety of model systems.

Related papers

• Hallmarks of Optimization Trajectories in Neural Networks: Directional Exploration and Redundancy [75.15685966213832]
  We analyze the rich directional structure of optimization trajectories represented by their pointwise parameters. We show that training only scalar batchnorm parameters some while into training matches the performance of training the entire network.
  arXiv  Detail & Related papers  (2024-03-12T07:32:47Z)
• Adaptive Federated Learning Over the Air [108.62635460744109]
  We propose a federated version of adaptive gradient methods, particularly AdaGrad and Adam, within the framework of over-the-air model training. Our analysis shows that the AdaGrad-based training algorithm converges to a stationary point at the rate of $mathcalO( ln(T) / T 1 - frac1alpha ).
  arXiv  Detail & Related papers  (2024-03-11T09:10:37Z)
• DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems [22.62539147446799]
  Learning dynamics from dissipative chaotic systems is notoriously difficult due to their inherent instability. We propose a new framework that targets learning the invariant measure as well as the dynamics. By targeting the distribution with a scalable regularization term, we hope that this approach can be extended to more complex systems.
  arXiv  Detail & Related papers  (2024-02-06T23:26:12Z)
• Stochastic Latent Transformer: Efficient Modelling of Stochastically Forced Zonal Jets [0.0]
  We present a novel deep probabilistic learning approach, the 'Stochastic Latent Transformer' (SLT) The SLT accurately reproduces system dynamics across various integration periods, validated through quantitative diagnostics. It achieves a five-order-of-magnitude speedup in emulating the zonally-averaged flow.
  arXiv  Detail & Related papers  (2023-10-25T16:17:00Z)
• Loss Dynamics of Temporal Difference Reinforcement Learning [36.772501199987076]
  We study the case learning curves for temporal difference learning of a value function with linear function approximators. We study how learning dynamics and plateaus depend on feature structure, learning rate, discount factor, and reward function.
  arXiv  Detail & Related papers  (2023-07-10T18:17:50Z)
• Training neural operators to preserve invariant measures of chaotic attractors [10.61157131995679]
  We show that a contrastive learning framework can preserve statistical properties of the dynamics nearly as well as the optimal transport approach. Our method is shown empirically to preserve invariant measures of chaotic attractors.
  arXiv  Detail & Related papers  (2023-06-01T22:56:30Z)
• Guaranteed Conservation of Momentum for Learning Particle-based Fluid Dynamics [96.9177297872723]
  We present a novel method for guaranteeing linear momentum in learned physics simulations. We enforce conservation of momentum with a hard constraint, which we realize via antisymmetrical continuous convolutional layers. In combination, the proposed method allows us to increase the physical accuracy of the learned simulator substantially.
  arXiv  Detail & Related papers  (2022-10-12T09:12:59Z)
• Manifold Interpolating Optimal-Transport Flows for Trajectory Inference [64.94020639760026]
  We present a method called Manifold Interpolating Optimal-Transport Flow (MIOFlow) MIOFlow learns, continuous population dynamics from static snapshot samples taken at sporadic timepoints. We evaluate our method on simulated data with bifurcations and merges, as well as scRNA-seq data from embryoid body differentiation, and acute myeloid leukemia treatment.
  arXiv  Detail & Related papers  (2022-06-29T22:19:03Z)
• Convex Analysis of the Mean Field Langevin Dynamics [49.66486092259375]
  convergence rate analysis of the mean field Langevin dynamics is presented. $p_q$ associated with the dynamics allows us to develop a convergence theory parallel to classical results in convex optimization.
  arXiv  Detail & Related papers  (2022-01-25T17:13:56Z)
• Training Generative Adversarial Networks by Solving Ordinary Differential Equations [54.23691425062034]
  We study the continuous-time dynamics induced by GAN training. From this perspective, we hypothesise that instabilities in training GANs arise from the integration error. We experimentally verify that well-known ODE solvers (such as Runge-Kutta) can stabilise training.
  arXiv  Detail & Related papers  (2020-10-28T15:23:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.