Nonlinear Assimilation via Score-based Sequential Langevin Sampling

• URL: http://arxiv.org/abs/2411.13443v2
• Date: Sun, 23 Feb 2025 12:34:08 GMT
• Title: Nonlinear Assimilation via Score-based Sequential Langevin Sampling
• Authors: Zhao Ding, Chenguang Duan, Yuling Jiao, Jerry Zhijian Yang, Cheng Yuan, Pingwen Zhang,
• Abstract summary: This paper presents score-based sequential Langevin sampling (SSLS)<n>The proposed method decomposes the assimilation process into alternating prediction and update steps.<n>We provide theoretical guarantees for SSLS convergence in total variation (TV) distance under certain conditions.
• Score: 5.107329143106734
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents score-based sequential Langevin sampling (SSLS), a novel approach to nonlinear data assimilation within a recursive Bayesian framework. The proposed method decomposes the assimilation process into alternating prediction and update steps, leveraging dynamic models for state prediction while incorporating observational data through score-based Langevin Monte Carlo during updates. To address challenges in posterior sampling, we introduce an annealing strategy within the update mechanism. We provide theoretical guarantees for SSLS convergence in total variation (TV) distance under certain conditions, providing insights into error behavior with respect to key hyper-parameters. Our numerical experiments across challenging scenarios -- including high-dimensional systems, strong nonlinearity, and sparse observations -- demonstrate the robust performance of the proposed method. Furthermore, SSLS effectively quantifies the uncertainty associated with the estimated states, making it particularly valuable for the error calibration.

Related papers

• Nonlinear Bayesian Update via Ensemble Kernel Regression with Clustering and Subsampling [0.87024326813104]
  We propose to extend traditional ensemble Kalman filtering to settings characterized by non-Gaussian priors and nonlinear measurement operators. In this framework, the observed component is first denoised via a standard Kalman update, while the unobserved component is estimated using a nonlinear regression approach.
  arXiv  Detail & Related papers  (2025-03-19T12:35:28Z)
• Constrained Sampling with Primal-Dual Langevin Monte Carlo [15.634831573546041]
  This work considers the problem of sampling from a probability distribution known up to a normalization constant. It satisfies a set of statistical constraints specified by the expected values of general nonlinear functions. We put forward a discrete-time primal-dual Langevin Monte Carlo algorithm (PD-LMC) that simultaneously constrains the target distribution and samples from it.
  arXiv  Detail & Related papers  (2024-11-01T13:26:13Z)
• SMR: State Memory Replay for Long Sequence Modeling [19.755738298836526]
  This paper proposes a novel non-recursive non-uniform sample processing strategy to overcome compatibility limitations in parallel convolutional computation. We introduce State Memory Replay (SMR), which utilizes learnable memories to adjust the current state with multi-step information for generalization at sampling points different from those in the training data. Experiments on long-range modeling tasks in autoregressive language modeling and Long Range Arena demonstrate the general effectiveness of the SMR mechanism for a series of SSM models.
  arXiv  Detail & Related papers  (2024-05-27T17:53:32Z)
• Causal Inference from Slowly Varying Nonstationary Processes [2.3072402651280517]
  Causal inference from observational data hinges on asymmetry between cause and effect from the data generating mechanisms. We propose a new class of restricted structural causal models, via a time-varying filter and stationary noise, and exploit the asymmetry from nonstationarity for causal identification.
  arXiv  Detail & Related papers  (2024-05-11T04:15:47Z)
• A Poisson-Gamma Dynamic Factor Model with Time-Varying Transition Dynamics [51.147876395589925]
  A non-stationary PGDS is proposed to allow the underlying transition matrices to evolve over time. A fully-conjugate and efficient Gibbs sampler is developed to perform posterior simulation. Experiments show that, in comparison with related models, the proposed non-stationary PGDS achieves improved predictive performance.
  arXiv  Detail & Related papers  (2024-02-26T04:39:01Z)
• Online Variational Sequential Monte Carlo [49.97673761305336]
  We build upon the variational sequential Monte Carlo (VSMC) method, which provides computationally efficient and accurate model parameter estimation and Bayesian latent-state inference. Online VSMC is capable of performing efficiently, entirely on-the-fly, both parameter estimation and particle proposal adaptation.
  arXiv  Detail & Related papers  (2023-12-19T21:45:38Z)
• Kalman Filter for Online Classification of Non-Stationary Data [101.26838049872651]
  In Online Continual Learning (OCL) a learning system receives a stream of data and sequentially performs prediction and training steps. We introduce a probabilistic Bayesian online learning model by using a neural representation and a state space model over the linear predictor weights. In experiments in multi-class classification we demonstrate the predictive ability of the model and its flexibility to capture non-stationarity.
  arXiv  Detail & Related papers  (2023-06-14T11:41:42Z)
• Convergence of uncertainty estimates in Ensemble and Bayesian sparse model discovery [4.446017969073817]
  We show empirical success in terms of accuracy and robustness to noise with bootstrapping-based sequential thresholding least-squares estimator. We show that this bootstrapping-based ensembling technique can perform a provably correct variable selection procedure with an exponential convergence rate of the error rate.
  arXiv  Detail & Related papers  (2023-01-30T04:07:59Z)
• Uncertainty Quantification for Traffic Forecasting: A Unified Approach [21.556559649467328]
  Uncertainty is an essential consideration for time series forecasting tasks. In this work, we focus on quantifying the uncertainty of traffic forecasting. We develop Deep S-Temporal Uncertainty Quantification (STUQ), which can estimate both aleatoric and relational uncertainty.
  arXiv  Detail & Related papers  (2022-08-11T15:21:53Z)
• Pessimistic Q-Learning for Offline Reinforcement Learning: Towards Optimal Sample Complexity [51.476337785345436]
  We study a pessimistic variant of Q-learning in the context of finite-horizon Markov decision processes. A variance-reduced pessimistic Q-learning algorithm is proposed to achieve near-optimal sample complexity.
  arXiv  Detail & Related papers  (2022-02-28T15:39:36Z)
• Time varying regression with hidden linear dynamics [74.9914602730208]
  We revisit a model for time-varying linear regression that assumes the unknown parameters evolve according to a linear dynamical system. Counterintuitively, we show that when the underlying dynamics are stable the parameters of this model can be estimated from data by combining just two ordinary least squares estimates.
  arXiv  Detail & Related papers  (2021-12-29T23:37:06Z)
• Likelihood-Free Inference in State-Space Models with Unknown Dynamics [71.94716503075645]
  We introduce a method for inferring and predicting latent states in state-space models where observations can only be simulated, and transition dynamics are unknown. We propose a way of doing likelihood-free inference (LFI) of states and state prediction with a limited number of simulations.
  arXiv  Detail & Related papers  (2021-11-02T12:33:42Z)
• On the Double Descent of Random Features Models Trained with SGD [78.0918823643911]
  We study properties of random features (RF) regression in high dimensions optimized by gradient descent (SGD) We derive precise non-asymptotic error bounds of RF regression under both constant and adaptive step-size SGD setting. We observe the double descent phenomenon both theoretically and empirically.
  arXiv  Detail & Related papers  (2021-10-13T17:47:39Z)
• Divergence Frontiers for Generative Models: Sample Complexity, Quantization Level, and Frontier Integral [58.434753643798224]
  Divergence frontiers have been proposed as an evaluation framework for generative models. We establish non-asymptotic bounds on the sample complexity of the plug-in estimator of divergence frontiers. We also augment the divergence frontier framework by investigating the statistical performance of smoothed distribution estimators.
  arXiv  Detail & Related papers  (2021-06-15T06:26:25Z)
• A Nonconvex Framework for Structured Dynamic Covariance Recovery [24.471814126358556]
  We propose a flexible yet interpretable model for high-dimensional data with time-varying second order statistics. Motivated by the literature, we quantify factorization and smooth temporal data. We show that our approach outperforms existing baselines.
  arXiv  Detail & Related papers  (2020-11-11T07:09:44Z)
• Stochastically forced ensemble dynamic mode decomposition for forecasting and analysis of near-periodic systems [65.44033635330604]
  We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system. We show that its use of intrinsic linear dynamics offers a number of desirable properties in terms of interpretability and parsimony. Results are presented for a test case using load data from an electrical grid.
  arXiv  Detail & Related papers  (2020-10-08T20:25:52Z)
• Semi-Supervised Empirical Risk Minimization: Using unlabeled data to improve prediction [4.860671253873579]
  We present a general methodology for using unlabeled data to design semi supervised learning (SSL) variants of the Empirical Risk Minimization (ERM) learning process. We analyze of the effectiveness of our SSL approach in improving prediction performance.
  arXiv  Detail & Related papers  (2020-09-01T17:55:51Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)
• LSTM-based Anomaly Detection for Non-linear Dynamical System [11.797156206007612]
  We propose a novel anomaly detection scheme in non-linear dynamical system based on Long Short-Term Memory (LSTM) We first present the framework of LSTM-based anomaly detection in non-linear dynamical system, including data preprocessing, multi-step prediction and anomaly detection. Our proposed multi-step prediction method can achieve a higher prediction accuracy than traditional method in wall shear stress dataset.
  arXiv  Detail & Related papers  (2020-06-05T01:09:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.