A Continuous-Time Ensemble Kalman-Bucy Smoother for Causal Inference and Model Discovery

• URL: http://arxiv.org/abs/2604.25157v2
• Date: Sun, 03 May 2026 07:32:28 GMT
• Title: A Continuous-Time Ensemble Kalman-Bucy Smoother for Causal Inference and Model Discovery
• Abstract summary: This paper introduces an ensemble Kalman-Bucy smoother (EnKBS) for continuous-time DA of nonlinear dynamical systems.<n>The skill of the EnKBS is demonstrated in various important scientific problems.
• Score: 6.75658113372928
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data assimilation (DA) integrates observational information with model predictions to improve state estimation in complex systems. While filtering provides the basis for online forecasts by using only past and present observations, it can exhibit delays and biases when the underlying dynamics evolve rapidly or undergo regime transitions. Smoothing, which additionally incorporates future observations, provides a natural pipeline for hindcasting and reanalysis that yields an uncertainty reduction beyond the filter. This paper introduces an ensemble Kalman-Bucy smoother (EnKBS) for continuous-time DA of nonlinear dynamical systems, where the smoother's conditional distributions are reconstructed using ensemble moments. The result is a derivative-free framework that does not require explicit computation of tangent-linear or adjoint models, which converges to the exact smoother solution at the infinite-ensemble limit for a wide class of complex systems. Incorporating standard regularization techniques for high-dimensional systems, such as covariance localization and inflation, the skill of the EnKBS is demonstrated in various important scientific problems. By integrating future observations, which reveal the underlying causal mechanisms for retrospective state updates, the EnKBS is used for Bayesian-based inference of causal relationships and their temporal influence range in a dyadic trigger-feedback model and the development of a causality-driven iterative learning algorithm that identifies the structure and recovers the hidden parameters of a nonlinear reduced-order model mimicking midlatitude atmospheric circulation. Notably, both tasks remain effective with an ensemble size of $O(10)$ under partial observations, suggesting that EnKBS can support the instantaneous discovery of high-dimensional complex systems over time.

Related papers

• Learning Spatio-Temporal Foundation Models from Pure Synthetic Data [85.23980441738811]
  NeoST is a scalable synthetic-training corpus to real-world bias, a latent-space reasoning architecture that generates and iteratively refines multiple future trajectories, and latent-space objectives that emphasize structural dynamics and enable inference-time correction under distribution shifts.<n>Extensive experiments across diverse real-world benchmarks show that NeoST consistently outperforms existing STFMs in diverse real-worldtemporal systems.
  arXiv  Detail & Related papers  (2026-06-27T02:56:46Z)
• Nonlocal Bayesian Modeling of Continuous Spatio-Temporal Dynamics [9.926231893220061]
  We propose a principled NonLocal S-NL model (BST) for continuous-temporal fields that learns nonlocal coupling while retaining tractable inference.<n> NLBST represents the latent field via a coordinate-based spatial basis expansion and models the sequential process with a continuous-time ODE.<n>A linear-Gssianau observation model enables Kalmanssian-style sequential sparse updates under missing and irregular observations, while the spatial basis representation enables inductive prediction at unmeasured locations without retraining.
  arXiv  Detail & Related papers  (2026-06-12T09:51:07Z)
• Improving Spatio-Temporal Residual Error Propagation by Mitigating Over-Squashing [4.62541204592513]
  We introduce Teger, a structured uncertainty module that overcomes the spa- tial and temporal limitations of error-related autoregressive forecasting.<n>Teger is backbone-agnostic, requiring only the latent state produced by any autoregressive encoder.
  arXiv  Detail & Related papers  (2026-05-18T08:51:33Z)
• PnP-Corrector: A Universal Correction Framework for Coupled Spatiotemporal Forecasting [62.22445401483844]
  Existing methods are severely constrained by the bottleneck of compounding errors.<n>We propose a universal framework called -Cast-Corrector (Plug-and-Play Corrector)<n>Our method significantly enhances the long-term stability of coupled forecasting systems.
  arXiv  Detail & Related papers  (2026-05-09T13:12:33Z)
• GeodesicNVS: Probability Density Geodesic Flow Matching for Novel View Synthesis [54.39598154430305]
  We propose a Data-to-Data Flow Matching framework that learns deterministic transformations directly between paired views.<n>PDG-FM constrains flow trajectories using geodesic interpolants derived from probability density metrics of pretrained diffusion models.<n>These results highlight the advantages of incorporating data-dependent geometric regularization into deterministic flow matching for consistent novel view generation.
  arXiv  Detail & Related papers  (2026-03-01T09:30:11Z)
• KoopGen: Koopman Generator Networks for Representing and Predicting Dynamical Systems with Continuous Spectra [65.11254608352982]
  We introduce a generator-based neural Koopman framework that models dynamics through a structured, state-dependent representation of Koopman generators.<n>By exploiting the intrinsic Cartesian decomposition into skew-adjoint and self-adjoint components, KoopGen separates conservative transport from irreversible dissipation.
  arXiv  Detail & Related papers  (2026-02-15T06:32:23Z)
• The Procrustean Bed of Time Series: The Optimization Bias of Point-wise Loss [53.542743390809356]
  This paper aims to provide a first-principles analysis of the Expectation of Optimization Bias (EOB)<n>Our analysis reveals a fundamental paradigm paradox: the more deterministic and structured the time series, the more severe the bias by point-wise loss function.<n>We present a concrete solution that simultaneously achieves both principles via DFT or DWT.
  arXiv  Detail & Related papers  (2025-12-21T06:08:22Z)
• Learning solution operator of dynamical systems with diffusion maps kernel ridge regression [2.7802667650114485]
  We show that a simple kernel ridge regression (KRR) framework provides a strong baseline for long-term prediction of complex dynamical systems.<n>Across a broad range of systems, DM-KRR consistently outperforms state-of-the-art random feature, neural-network and operator-learning methods in both accuracy and data efficiency.
  arXiv  Detail & Related papers  (2025-12-19T03:29:23Z)
• Calibrating Geophysical Predictions under Constrained Probabilistic Distributions [4.760743517243988]
  We introduce a calibration algorithm based on normalization and the Kernelized Stein Discrepancy (KSD) to enhance machine learning predictions.<n>This not only sharpens pointwise predictions but also enforces consistency with non-local statistical structures rooted in physical principles.
  arXiv  Detail & Related papers  (2025-11-28T07:15:40Z)
• Learning-Enhanced Observer for Linear Time-Invariant Systems with Parametric Uncertainty [1.6498361958317636]
  This work introduces a learning-enhanced observer (LEO) for linear time-invariant systems with uncertain dynamics.<n>Rather than relying solely on nominal models, the proposed framework treats the system matrices as optimizable variables and refines them through gradient-based minimization of a steady-state output discrepancy loss.<n>The resulting data-informed surrogate model enables the construction of an improved observer that effectively compensates for moderate parameter uncertainty while preserving the structure of classical designs.
  arXiv  Detail & Related papers  (2025-11-20T12:47:52Z)
• Predicting Forced Responses of Probability Distributions via the Fluctuation-Dissipation Theorem and Generative Modeling [0.0]
  We present a data-driven framework for estimating the response of higher-order moments of nonlinear systems to small external perturbations.<n>We combine GFDT with score-based generative modeling to estimate the system's score function directly from data.<n>Our method is validated on several models relevant to climate dynamics.
  arXiv  Detail & Related papers  (2025-04-17T20:54:33Z)
• A Poisson-Gamma Dynamic Factor Model with Time-Varying Transition Dynamics [51.147876395589925]
  A non-stationary PGDS is proposed to allow the underlying transition matrices to evolve over time. A fully-conjugate and efficient Gibbs sampler is developed to perform posterior simulation. Experiments show that, in comparison with related models, the proposed non-stationary PGDS achieves improved predictive performance.
  arXiv  Detail & Related papers  (2024-02-26T04:39:01Z)
• Causal Temporal Regime Structure Learning [49.77103348208835]
  We present CASTOR, a novel method that concurrently learns the Directed Acyclic Graph (DAG) for each regime.<n>We establish the identifiability of the regimes and DAGs within our framework.<n>Experiments show that CASTOR consistently outperforms existing causal discovery models.
  arXiv  Detail & Related papers  (2023-11-02T17:26:49Z)
• Stochastically forced ensemble dynamic mode decomposition for forecasting and analysis of near-periodic systems [65.44033635330604]
  We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system. We show that its use of intrinsic linear dynamics offers a number of desirable properties in terms of interpretability and parsimony. Results are presented for a test case using load data from an electrical grid.
  arXiv  Detail & Related papers  (2020-10-08T20:25:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.