The Deep Latent Space Particle Filter for Real-Time Data Assimilation with Uncertainty Quantification

• URL: http://arxiv.org/abs/2406.02204v1
• Date: Tue, 4 Jun 2024 10:59:54 GMT
• Title: The Deep Latent Space Particle Filter for Real-Time Data Assimilation with Uncertainty Quantification
• Authors: Nikolaj T. Mücke, Sander M. Bohté, Cornelis W. Oosterlee,
• Abstract summary: We present a novel particle filter methodology, the Deep Latent Space Particle filter or D-LSPF, that uses neural network-based surrogate models to overcome this computational challenge. The D-LSPF runs orders of magnitude faster than a high-fidelity particle filter and 3-5 times faster than alternative methods while being up to an order of magnitude more accurate.
• Score: 0.669618059970013
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In Data Assimilation, observations are fused with simulations to obtain an accurate estimate of the state and parameters for a given physical system. Combining data with a model, however, while accurately estimating uncertainty, is computationally expensive and infeasible to run in real-time for complex systems. Here, we present a novel particle filter methodology, the Deep Latent Space Particle filter or D-LSPF, that uses neural network-based surrogate models to overcome this computational challenge. The D-LSPF enables filtering in the low-dimensional latent space obtained using Wasserstein AEs with modified vision transformer layers for dimensionality reduction and transformers for parameterized latent space time stepping. As we demonstrate on three test cases, including leak localization in multi-phase pipe flow and seabed identification for fully nonlinear water waves, the D-LSPF runs orders of magnitude faster than a high-fidelity particle filter and 3-5 times faster than alternative methods while being up to an order of magnitude more accurate. The D-LSPF thus enables real-time data assimilation with uncertainty quantification for physical systems.

Related papers

• Closed-form Filtering for Non-linear Systems [83.91296397912218]
  We propose a new class of filters based on Gaussian PSD Models, which offer several advantages in terms of density approximation and computational efficiency. We show that filtering can be efficiently performed in closed form when transitions and observations are Gaussian PSD Models. Our proposed estimator enjoys strong theoretical guarantees, with estimation error that depends on the quality of the approximation and is adaptive to the regularity of the transition probabilities.
  arXiv  Detail & Related papers  (2024-02-15T08:51:49Z)
• Learning Differentiable Particle Filter on the Fly [18.466658684464598]
  Differentiable particle filters are an emerging class of sequential Bayesian inference techniques. We propose an online learning framework for differentiable particle filters so that model parameters can be updated as data arrive.
  arXiv  Detail & Related papers  (2023-12-10T17:54:40Z)
• An Ensemble Score Filter for Tracking High-Dimensional Nonlinear Dynamical Systems [12.36064367319084]
  A major drawback of existing filtering methods is the low accuracy in handling high-dimensional and highly nonlinear problems. We propose an ensemble score filter (EnSF) for solving high-dimensional nonlinear filtering problems with superior accuracy. EnSF provides surprisingly impressive performance in reliably tracking extremely high-dimensional Lorenz systems.
  arXiv  Detail & Related papers  (2023-09-02T16:48:02Z)
• Learning IMM Filter Parameters from Measurements using Gradient Descent [45.335821132209766]
  In intrinsic parameters of targets under track can be completely unobservable until the system is deployed. With state-of-the-art sensor systems growing more and more complex, the number of parameters naturally increases. In this paper, the parameters of an interacting multiple model (IMM) filter are optimized solely using measurements.
  arXiv  Detail & Related papers  (2023-07-13T08:35:40Z)
• Filter Pruning for Efficient CNNs via Knowledge-driven Differential Filter Sampler [103.97487121678276]
  Filter pruning simultaneously accelerates the computation and reduces the memory overhead of CNNs. We propose a novel Knowledge-driven Differential Filter Sampler(KDFS) with Masked Filter Modeling(MFM) framework for filter pruning.
  arXiv  Detail & Related papers  (2023-07-01T02:28:41Z)
• Dynamic Temporal Filtering in Video Models [128.02725199486719]
  We present a new recipe of temporal feature learning, namely Dynamic Temporal Filter (DTF) DTF learns a specialized frequency filter for every spatial location to model its long-range temporal dynamics. It is feasible to plug DTF block into ConvNets and Transformer, yielding DTF-Net and DTF-Transformer.
  arXiv  Detail & Related papers  (2022-11-15T15:59:28Z)
• Computational Doob's h-transforms for Online Filtering of Discretely Observed Diffusions [65.74069050283998]
  We propose a computational framework to approximate Doob's $h$-transforms. The proposed approach can be orders of magnitude more efficient than state-of-the-art particle filters.
  arXiv  Detail & Related papers  (2022-06-07T15:03:05Z)
• An efficient estimation of time-varying parameters of dynamic models by combining offline batch optimization and online data assimilation [0.0]
  I present an efficient and practical method to estimate the time-varying parameters of relatively low dimensional models. I propose combining offline batch optimization and online data assimilation.
  arXiv  Detail & Related papers  (2021-10-24T20:12:12Z)
• Adaptive Machine Learning for Time-Varying Systems: Low Dimensional Latent Space Tuning [91.3755431537592]
  We present a recently developed method of adaptive machine learning for time-varying systems. Our approach is to map very high (N>100k) dimensional inputs into the low dimensional (N2) latent space at the output of the encoder section of an encoder-decoder CNN. This method allows us to learn correlations within and to track their evolution in real time based on feedback without interrupts.
  arXiv  Detail & Related papers  (2021-07-13T16:05:28Z)
• Adaptive Latent Space Tuning for Non-Stationary Distributions [62.997667081978825]
  We present a method for adaptive tuning of the low-dimensional latent space of deep encoder-decoder style CNNs. We demonstrate our approach for predicting the properties of a time-varying charged particle beam in a particle accelerator.
  arXiv  Detail & Related papers  (2021-05-08T03:50:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.