A Fokker-Planck-Based Loss Function that Bridges Dynamics with Density Estimation

• URL: http://arxiv.org/abs/2502.17690v2
• Date: Thu, 27 Feb 2025 22:11:09 GMT
• Title: A Fokker-Planck-Based Loss Function that Bridges Dynamics with Density Estimation
• Authors: Zhixin Lu, Łukasz Kuśmierz, Stefan Mihalas,
• Abstract summary: We derive a novel loss function from the Fokker-Planck equation that links dynamical system models with their probability density functions.<n>For density estimation, we propose a density estimator that integrates a Gaussian Mixture Model with a normalizing flow model.<n>It is compatible with a variety of data-based training methodologies, including maximum likelihood and score matching.
• Score: 1.8434042562191815
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We have derived a novel loss function from the Fokker-Planck equation that links dynamical system models with their probability density functions, demonstrating its utility in model identification and density estimation. In the first application, we show that this loss function can enable the extraction of dynamical parameters from non-temporal datasets, including timestamp-free measurements from steady non-equilibrium systems such as noisy Lorenz systems and gene regulatory networks. In the second application, when coupled with a density estimator, this loss facilitates density estimation when the dynamic equations are known. For density estimation, we propose a density estimator that integrates a Gaussian Mixture Model with a normalizing flow model. It simultaneously estimates normalized density, energy, and score functions from both empirical data and dynamics. It is compatible with a variety of data-based training methodologies, including maximum likelihood and score matching. It features a latent space akin to a modern Hopfield network, where the inherent Hopfield energy effectively assigns low densities to sparsely populated data regions, addressing common challenges in neural density estimators. Additionally, this Hopfield-like energy enables direct and rapid data manipulation through the Concave-Convex Procedure (CCCP) rule, facilitating tasks such as denoising and clustering. Our work demonstrates a principled framework for leveraging the complex interdependencies between dynamics and density estimation, as illustrated through synthetic examples that clarify the underlying theoretical intuitions.

Related papers

• Density Estimation via Binless Multidimensional Integration [45.21975243399607]
  We introduce the Binless Multidimensional Thermodynamic Integration (BMTI) method for nonparametric, robust, and data-efficient density estimation. BMTI estimates the logarithm of the density by initially computing log-density differences between neighbouring data points. The method is tested on a variety of complex synthetic high-dimensional datasets, and is benchmarked on realistic datasets from the chemical physics literature.
  arXiv  Detail & Related papers  (2024-07-10T23:45:20Z)
• Dynamical Measure Transport and Neural PDE Solvers for Sampling [77.38204731939273]
  We tackle the task of sampling from a probability density as transporting a tractable density function to the target. We employ physics-informed neural networks (PINNs) to approximate the respective partial differential equations (PDEs) solutions. PINNs allow for simulation- and discretization-free optimization and can be trained very efficiently.
  arXiv  Detail & Related papers  (2024-07-10T17:39:50Z)
• Regulating Model Reliance on Non-Robust Features by Smoothing Input Marginal Density [93.32594873253534]
  Trustworthy machine learning requires meticulous regulation of model reliance on non-robust features. We propose a framework to delineate and regulate such features by attributing model predictions to the input.
  arXiv  Detail & Related papers  (2024-07-05T09:16:56Z)
• On the Trajectory Regularity of ODE-based Diffusion Sampling [79.17334230868693]
  Diffusion-based generative models use differential equations to establish a smooth connection between a complex data distribution and a tractable prior distribution. In this paper, we identify several intriguing trajectory properties in the ODE-based sampling process of diffusion models.
  arXiv  Detail & Related papers  (2024-05-18T15:59:41Z)
• Adaptive deep density approximation for stochastic dynamical systems [0.5120567378386615]
  A new temporal KRnet is proposed to approximate the probability density functions (PDFs) iteration of the state variables. To efficiently train the tKRnet, an adaptive procedure is developed to generate collocation points for the corresponding residual loss function. A temporal decomposition technique is also employed to improve the long-time integration.
  arXiv  Detail & Related papers  (2024-05-05T04:29:22Z)
• Topological Detection of Phenomenological Bifurcations with Unreliable Kernel Densities [0.5874142059884521]
  Phenomenological (P-type) bifurcations are qualitative changes in dynamical systems. Current state of the art for detecting these bifurcations requires reliable kernel density estimates computed from an ensemble of system realizations. This study presents an approach for detecting P-type bifurcations using unreliable density estimates.
  arXiv  Detail & Related papers  (2024-01-29T20:59:25Z)
• Capturing dynamical correlations using implicit neural representations [85.66456606776552]
  We develop an artificial intelligence framework which combines a neural network trained to mimic simulated data from a model Hamiltonian with automatic differentiation to recover unknown parameters from experimental data. In doing so, we illustrate the ability to build and train a differentiable model only once, which then can be applied in real-time to multi-dimensional scattering data.
  arXiv  Detail & Related papers  (2023-04-08T07:55:36Z)
• Convex Analysis of the Mean Field Langevin Dynamics [49.66486092259375]
  convergence rate analysis of the mean field Langevin dynamics is presented. $p_q$ associated with the dynamics allows us to develop a convergence theory parallel to classical results in convex optimization.
  arXiv  Detail & Related papers  (2022-01-25T17:13:56Z)
• Density-Based Clustering with Kernel Diffusion [59.4179549482505]
  A naive density corresponding to the indicator function of a unit $d$-dimensional Euclidean ball is commonly used in density-based clustering algorithms. We propose a new kernel diffusion density function, which is adaptive to data of varying local distributional characteristics and smoothness.
  arXiv  Detail & Related papers  (2021-10-11T09:00:33Z)
• A data-driven peridynamic continuum model for upscaling molecular dynamics [3.1196544696082613]
  We propose a learning framework to extract, from molecular dynamics data, an optimal Linear Peridynamic Solid model. We provide sufficient well-posedness conditions for discretized LPS models with sign-changing influence functions. This framework guarantees that the resulting model is mathematically well-posed, physically consistent, and that it generalizes well to settings that are different from the ones used during training.
  arXiv  Detail & Related papers  (2021-08-04T07:07:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.