Reinforcement learning based data assimilation for unknown state model

• URL: http://arxiv.org/abs/2511.02286v1
• Date: Tue, 04 Nov 2025 05:58:37 GMT
• Title: Reinforcement learning based data assimilation for unknown state model
• Authors: Ziyi Wang, Lijian Jiang,
• Abstract summary: We propose a novel method that integrates reinforcement learning with ensemble-based Bayesian ffltering methods.<n>The proposed framework accommodates a wide range of observation scenarios, including nonlinear and partially observed measurement models.<n>A few numerical examples demonstrate that the proposed method achieves superior accuracy and robustness in high-dimensional settings.
• Score: 3.032674692886751
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Data assimilation (DA) has increasingly emerged as a critical tool for state estimation across a wide range of applications. It is signiffcantly challenging when the governing equations of the underlying dynamics are unknown. To this end, various machine learning approaches have been employed to construct a surrogate state transition model in a supervised learning framework, which relies on pre-computed training datasets. However, it is often infeasible to obtain noise-free ground-truth state sequences in practice. To address this challenge, we propose a novel method that integrates reinforcement learning with ensemble-based Bayesian ffltering methods, enabling the learning of surrogate state transition model for unknown dynamics directly from noisy observations, without using true state trajectories. Speciffcally, we treat the process for computing maximum likelihood estimation of surrogate model parameters as a sequential decision-making problem, which can be formulated as a discretetime Markov decision process (MDP). Under this formulation, learning the surrogate transition model is equivalent to ffnding an optimal policy of the MDP, which can be effectively addressed using reinforcement learning techniques. Once the model is trained offfine, state estimation can be performed in the online stage using ffltering methods based on the learned dynamics. The proposed framework accommodates a wide range of observation scenarios, including nonlinear and partially observed measurement models. A few numerical examples demonstrate that the proposed method achieves superior accuracy and robustness in high-dimensional settings.

Related papers

• Denoising Particle Filters: Learning State Estimation with Single-Step Objectives [2.320417845168326]
  We propose a novel particle filtering algorithm in which models are trained from individual state transitions.<n>We evaluate the proposed method on challenging robotic state estimation tasks in simulation.
  arXiv  Detail & Related papers  (2026-02-23T09:53:23Z)
• Semi-Supervised Model-Free Bayesian State Estimation from Compressed Measurements [57.04370580292727]
  We consider data-driven Bayesian state estimation from compressed measurements.<n>The dimension of the temporal measurement vector is lower than that of the temporal state vector to be estimated.<n>The underlying dynamical model of the state's evolution is unknown for a'model-free process'
  arXiv  Detail & Related papers  (2024-07-10T05:03:48Z)
• Towards Learning Stochastic Population Models by Gradient Descent [0.0]
  We show that simultaneous estimation of parameters and structure poses major challenges for optimization procedures. We demonstrate accurate estimation of models but find that enforcing the inference of parsimonious, interpretable models drastically increases the difficulty.
  arXiv  Detail & Related papers  (2024-04-10T14:38:58Z)
• Gaussian process learning of nonlinear dynamics [0.0]
  We propose a new method that learns nonlinear dynamics through a Bayesian inference of characterizing model parameters. We will discuss the applicability of the proposed method to several typical scenarios for dynamical systems.
  arXiv  Detail & Related papers  (2023-12-19T14:27:26Z)
• Distributionally Robust Model-based Reinforcement Learning with Large State Spaces [55.14361269378122]
  Three major challenges in reinforcement learning are the complex dynamical systems with large state spaces, the costly data acquisition processes, and the deviation of real-world dynamics from the training environment deployment. We study distributionally robust Markov decision processes with continuous state spaces under the widely used Kullback-Leibler, chi-square, and total variation uncertainty sets. We propose a model-based approach that utilizes Gaussian Processes and the maximum variance reduction algorithm to efficiently learn multi-output nominal transition dynamics.
  arXiv  Detail & Related papers  (2023-09-05T13:42:11Z)
• MACE: An Efficient Model-Agnostic Framework for Counterfactual Explanation [132.77005365032468]
  We propose a novel framework of Model-Agnostic Counterfactual Explanation (MACE) In our MACE approach, we propose a novel RL-based method for finding good counterfactual examples and a gradient-less descent method for improving proximity. Experiments on public datasets validate the effectiveness with better validity, sparsity and proximity.
  arXiv  Detail & Related papers  (2022-05-31T04:57:06Z)
• Data Augmentation through Expert-guided Symmetry Detection to Improve Performance in Offline Reinforcement Learning [0.0]
  offline estimation of the dynamical model of a Markov Decision Process (MDP) is a non-trivial task. Recent works showed that an expert-guided pipeline relying on Density Estimation methods effectively detects this structure in deterministic environments. We show that the former results lead to a performance improvement when solving the learned MDP and then applying the optimized policy in the real environment.
  arXiv  Detail & Related papers  (2021-12-18T14:32:32Z)
• MINIMALIST: Mutual INformatIon Maximization for Amortized Likelihood Inference from Sampled Trajectories [61.3299263929289]
  Simulation-based inference enables learning the parameters of a model even when its likelihood cannot be computed in practice. One class of methods uses data simulated with different parameters to infer an amortized estimator for the likelihood-to-evidence ratio. We show that this approach can be formulated in terms of mutual information between model parameters and simulated data.
  arXiv  Detail & Related papers  (2021-06-03T12:59:16Z)
• DEALIO: Data-Efficient Adversarial Learning for Imitation from Observation [57.358212277226315]
  In imitation learning from observation IfO, a learning agent seeks to imitate a demonstrating agent using only observations of the demonstrated behavior without access to the control signals generated by the demonstrator. Recent methods based on adversarial imitation learning have led to state-of-the-art performance on IfO problems, but they typically suffer from high sample complexity due to a reliance on data-inefficient, model-free reinforcement learning algorithms. This issue makes them impractical to deploy in real-world settings, where gathering samples can incur high costs in terms of time, energy, and risk. We propose a more data-efficient IfO algorithm
  arXiv  Detail & Related papers  (2021-03-31T23:46:32Z)
• Learning Unstable Dynamics with One Minute of Data: A Differentiation-based Gaussian Process Approach [47.045588297201434]
  We show how to exploit the differentiability of Gaussian processes to create a state-dependent linearized approximation of the true continuous dynamics. We validate our approach by iteratively learning the system dynamics of an unstable system such as a 9-D segway.
  arXiv  Detail & Related papers  (2021-03-08T05:08:47Z)
• Model-Augmented Actor-Critic: Backpropagating through Paths [81.86992776864729]
  Current model-based reinforcement learning approaches use the model simply as a learned black-box simulator. We show how to make more effective use of the model by exploiting its differentiability.
  arXiv  Detail & Related papers  (2020-05-16T19:18:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.