Real-Time Model Calibration with Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2006.04001v2
• Date: Tue, 9 Jun 2020 12:25:49 GMT
• Title: Real-Time Model Calibration with Deep Reinforcement Learning
• Authors: Yuan Tian, Manuel Arias Chao, Chetan Kulkarni, Kai Goebel and Olga Fink
• Abstract summary: We propose a novel framework for inference of model parameters based on reinforcement learning. The proposed methodology is demonstrated and evaluated on two model-based diagnostics test cases.
• Score: 4.707841918805165
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The dynamic, real-time, and accurate inference of model parameters from empirical data is of great importance in many scientific and engineering disciplines that use computational models (such as a digital twin) for the analysis and prediction of complex physical processes. However, fast and accurate inference for processes with large and high dimensional datasets cannot easily be achieved with state-of-the-art methods under noisy real-world conditions. The primary reason is that the inference of model parameters with traditional techniques based on optimisation or sampling often suffers from computational and statistical challenges, resulting in a trade-off between accuracy and deployment time. In this paper, we propose a novel framework for inference of model parameters based on reinforcement learning. The contribution of the paper is twofold: 1) We reformulate the inference problem as a tracking problem with the objective of learning a policy that forces the response of the physics-based model to follow the observations; 2) We propose the constrained Lyapunov-based actor-critic (CLAC) algorithm to enable the robust and accurate inference of physics-based model parameters in real time under noisy real-world conditions. The proposed methodology is demonstrated and evaluated on two model-based diagnostics test cases utilizing two different physics-based models of turbofan engines. The performance of the methodology is compared to that of two alternative approaches: a state update method (unscented Kalman filter) and a supervised end-to-end mapping with deep neural networks. The experimental results demonstrate that the proposed methodology outperforms all other tested methods in terms of speed and robustness, with high inference accuracy.

Related papers

• Towards Learning Stochastic Population Models by Gradient Descent [0.0]
  We show that simultaneous estimation of parameters and structure poses major challenges for optimization procedures. We demonstrate accurate estimation of models but find that enforcing the inference of parsimonious, interpretable models drastically increases the difficulty.
  arXiv  Detail & Related papers  (2024-04-10T14:38:58Z)
• On the Effectiveness of Parameter-Efficient Fine-Tuning [79.6302606855302]
  Currently, many research works propose to only fine-tune a small portion of the parameters while keeping most of the parameters shared across different tasks. We show that all of the methods are actually sparse fine-tuned models and conduct a novel theoretical analysis of them. Despite the effectiveness of sparsity grounded by our theory, it still remains an open problem of how to choose the tunable parameters.
  arXiv  Detail & Related papers  (2022-11-28T17:41:48Z)
• Neural Superstatistics for Bayesian Estimation of Dynamic Cognitive Models [2.7391842773173334]
  We develop a simulation-based deep learning method for Bayesian inference, which can recover both time-varying and time-invariant parameters. Our results show that the deep learning approach is very efficient in capturing the temporal dynamics of the model.
  arXiv  Detail & Related papers  (2022-11-23T17:42:53Z)
• When to Update Your Model: Constrained Model-based Reinforcement Learning [50.74369835934703]
  We propose a novel and general theoretical scheme for a non-decreasing performance guarantee of model-based RL (MBRL) Our follow-up derived bounds reveal the relationship between model shifts and performance improvement. A further example demonstrates that learning models from a dynamically-varying number of explorations benefit the eventual returns.
  arXiv  Detail & Related papers  (2022-10-15T17:57:43Z)
• LEARNEST: LEARNing Enhanced Model-based State ESTimation for Robots using Knowledge-based Neural Ordinary Differential Equations [4.3403382998035624]
  In this work, we consider the task of obtaining accurate state estimates for robotic systems by enhancing the dynamics model used in state estimation algorithms. To enhance the dynamics models and improve the estimation accuracy, we utilize a deep learning framework known as knowledge-based neural ordinary differential equations (KNODEs) In our proposed LEARNEST framework, we integrate the data-driven model into two novel model-based state estimation algorithms, which are denoted as KNODE-MHE and KNODE-UKF.
  arXiv  Detail & Related papers  (2022-09-16T22:16:40Z)
• Human Trajectory Prediction via Neural Social Physics [63.62824628085961]
  Trajectory prediction has been widely pursued in many fields, and many model-based and model-free methods have been explored. We propose a new method combining both methodologies based on a new Neural Differential Equation model. Our new model (Neural Social Physics or NSP) is a deep neural network within which we use an explicit physics model with learnable parameters.
  arXiv  Detail & Related papers  (2022-07-21T12:11:18Z)
• Gradient-Based Trajectory Optimization With Learned Dynamics [80.41791191022139]
  We use machine learning techniques to learn a differentiable dynamics model of the system from data. We show that a neural network can model highly nonlinear behaviors accurately for large time horizons. In our hardware experiments, we demonstrate that our learned model can represent complex dynamics for both the Spot and Radio-controlled (RC) car.
  arXiv  Detail & Related papers  (2022-04-09T22:07:34Z)
• Learning new physics efficiently with nonparametric methods [11.970219534238444]
  We present a machine learning approach for model-independent new physics searches. The corresponding algorithm is powered by recent large-scale implementations of kernel methods. We show that our approach has dramatic advantages compared to neural network implementations in terms of training times and computational resources.
  arXiv  Detail & Related papers  (2022-04-05T16:17:59Z)
• Progressive residual learning for single image dehazing [57.651704852274825]
  A progressive residual learning strategy has been proposed to combine the physical model-free dehazing process with reformulated scattering model-based dehazing operations. The proposed method performs favorably against the state-of-the-art methods on public dehazing benchmarks with better model interpretability and adaptivity for complex data.
  arXiv  Detail & Related papers  (2021-03-14T16:54:44Z)
• Double Robust Representation Learning for Counterfactual Prediction [68.78210173955001]
  We propose a novel scalable method to learn double-robust representations for counterfactual predictions. We make robust and efficient counterfactual predictions for both individual and average treatment effects. The algorithm shows competitive performance with the state-of-the-art on real world and synthetic data.
  arXiv  Detail & Related papers  (2020-10-15T16:39:26Z)
• Modeling Stochastic Microscopic Traffic Behaviors: a Physics Regularized Gaussian Process Approach [1.6242924916178285]
  This study presents a microscopic traffic model that can capture randomness and measure errors in the real world. Since one unique feature of the proposed framework is the capability of capturing both car-following and lane-changing behaviors with one single model, numerical tests are carried out with two separated datasets.
  arXiv  Detail & Related papers  (2020-07-17T06:03:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.