Time-dependent density estimation using binary classifiers

• URL: http://arxiv.org/abs/2506.15505v1
• Date: Wed, 18 Jun 2025 14:43:04 GMT
• Title: Time-dependent density estimation using binary classifiers
• Authors: Agnimitra Dasgupta, Javier Murgoitio-Esandi, Ali Fardisi, Assad A Oberai,
• Abstract summary: We propose a data-driven method to learn the time-dependent probability density of a multivariate process from sample paths.<n>We show that the proposed method accurately reconstructs complex time-dependent, multi-dimensional, and near-degenerate densities, scales effectively to moderately high-dimensional problems, and reliably detects rare events among real-world data.
• Score: 0.22369578015657962
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a data-driven method to learn the time-dependent probability density of a multivariate stochastic process from sample paths, assuming that the initial probability density is known and can be evaluated. Our method uses a novel time-dependent binary classifier trained using a contrastive estimation-based objective that trains the classifier to discriminate between realizations of the stochastic process at two nearby time instants. Significantly, the proposed method explicitly models the time-dependent probability distribution, which means that it is possible to obtain the value of the probability density within the time horizon of interest. Additionally, the input before the final activation in the time-dependent classifier is a second-order approximation to the partial derivative, with respect to time, of the logarithm of the density. We apply the proposed approach to approximate the time-dependent probability density functions for systems driven by stochastic excitations. We also use the proposed approach to synthesize new samples of a random vector from a given set of its realizations. In such applications, we generate sample paths necessary for training using stochastic interpolants. Subsequently, new samples are generated using gradient-based Markov chain Monte Carlo methods because automatic differentiation can efficiently provide the necessary gradient. Further, we demonstrate the utility of an explicit approximation to the time-dependent probability density function through applications in unsupervised outlier detection. Through several numerical experiments, we show that the proposed method accurately reconstructs complex time-dependent, multi-modal, and near-degenerate densities, scales effectively to moderately high-dimensional problems, and reliably detects rare events among real-world data.

Related papers

• Sampling in High-Dimensions using Stochastic Interpolants and Forward-Backward Stochastic Differential Equations [8.509310102094512]
  We present a class of diffusion-based algorithms to draw samples from high-dimensional probability distributions.<n>Our approach relies on the interpolants framework to define a time-indexed collection of probability densities.<n>We demonstrate that our algorithm can effectively draw samples from distributions that conventional methods struggle to handle.
  arXiv  Detail & Related papers  (2025-02-01T07:27:11Z)
• Quasi-Bayesian sequential deconvolution [7.10052009802944]
  We develop a principled sequential approach to estimate $f$ in a streaming or online domain.<n>Local and uniform Gaussian central limit theorems for $f_n$ are established, leading to credible intervals and bands for $f$.<n>An empirical validation of our methods is presented on synthetic and real data.
  arXiv  Detail & Related papers  (2024-08-26T16:40:04Z)
• Dynamical Measure Transport and Neural PDE Solvers for Sampling [77.38204731939273]
  We tackle the task of sampling from a probability density as transporting a tractable density function to the target. We employ physics-informed neural networks (PINNs) to approximate the respective partial differential equations (PDEs) solutions. PINNs allow for simulation- and discretization-free optimization and can be trained very efficiently.
  arXiv  Detail & Related papers  (2024-07-10T17:39:50Z)
• DynGMA: a robust approach for learning stochastic differential equations from data [13.858051019755283]
  We introduce novel approximations to the transition density of the parameterized SDE. Our method exhibits superior accuracy compared to baseline methods in learning the fully unknown drift diffusion functions. It is capable of handling data with low time resolution and variable, even uncontrollable, time step sizes.
  arXiv  Detail & Related papers  (2024-02-22T12:09:52Z)
• Deep Ensembles Meets Quantile Regression: Uncertainty-aware Imputation for Time Series [45.76310830281876]
  We propose Quantile Sub-Ensembles, a novel method to estimate uncertainty with ensemble of quantile-regression-based task networks. Our method not only produces accurate imputations that is robust to high missing rates, but also is computationally efficient due to the fast training of its non-generative model.
  arXiv  Detail & Related papers  (2023-12-03T05:52:30Z)
• Stochastic Interpolants: A Unifying Framework for Flows and Diffusions [16.95541777254722]
  A class of generative models that unifies flow-based and diffusion-based methods is introduced. These models extend the framework proposed in Albergo & VandenEijnden (2023), enabling the use of a broad class of continuous-time processes called stochastic interpolants' These interpolants are built by combining data from the two prescribed densities with an additional latent variable that shapes the bridge in a flexible way.
  arXiv  Detail & Related papers  (2023-03-15T17:43:42Z)
• Adapting to Continuous Covariate Shift via Online Density Ratio Estimation [64.8027122329609]
  Dealing with distribution shifts is one of the central challenges for modern machine learning. We propose an online method that can appropriately reuse historical information. Our density ratio estimation method is proven to perform well by enjoying a dynamic regret bound.
  arXiv  Detail & Related papers  (2023-02-06T04:03:33Z)
• Score-based Continuous-time Discrete Diffusion Models [102.65769839899315]
  We extend diffusion models to discrete variables by introducing a Markov jump process where the reverse process denoises via a continuous-time Markov chain. We show that an unbiased estimator can be obtained via simple matching the conditional marginal distributions. We demonstrate the effectiveness of the proposed method on a set of synthetic and real-world music and image benchmarks.
  arXiv  Detail & Related papers  (2022-11-30T05:33:29Z)
• Probability flow solution of the Fokker-Planck equation [10.484851004093919]
  We introduce an alternative scheme based on integrating an ordinary differential equation that describes the flow of probability. Unlike the dynamics, this equation deterministically pushes samples from the initial density onto samples from the solution at any later time. Our approach is based on recent advances in score-based diffusion for generative modeling.
  arXiv  Detail & Related papers  (2022-06-09T17:37:09Z)
• Density Ratio Estimation via Infinitesimal Classification [85.08255198145304]
  We propose DRE-infty, a divide-and-conquer approach to reduce Density ratio estimation (DRE) to a series of easier subproblems. Inspired by Monte Carlo methods, we smoothly interpolate between the two distributions via an infinite continuum of intermediate bridge distributions. We show that our approach performs well on downstream tasks such as mutual information estimation and energy-based modeling on complex, high-dimensional datasets.
  arXiv  Detail & Related papers  (2021-11-22T06:26:29Z)
• Sampling from Arbitrary Functions via PSD Models [55.41644538483948]
  We take a two-step approach by first modeling the probability distribution and then sampling from that model. We show that these models can approximate a large class of densities concisely using few evaluations, and present a simple algorithm to effectively sample from these models.
  arXiv  Detail & Related papers  (2021-10-20T12:25:22Z)
• Contrastive learning of strong-mixing continuous-time stochastic processes [53.82893653745542]
  Contrastive learning is a family of self-supervised methods where a model is trained to solve a classification task constructed from unlabeled data. We show that a properly constructed contrastive learning task can be used to estimate the transition kernel for small-to-mid-range intervals in the diffusion case.
  arXiv  Detail & Related papers  (2021-03-03T23:06:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.