Peaking into the Black-box: Prediction Intervals Give Insight into Data-driven Quadrotor Model Reliability

• URL: http://arxiv.org/abs/2408.06036v1
• Date: Mon, 12 Aug 2024 09:57:00 GMT
• Title: Peaking into the Black-box: Prediction Intervals Give Insight into Data-driven Quadrotor Model Reliability
• Authors: Jasper van Beers, Coen de Visser,
• Abstract summary: Prediction intervals (PIs) may be employed to provide insight into consistency and accuracy of model predictions. This paper estimates such PIs for bootstrap and Artificial Neural Network (ANN) quadrotor aerodynamic models. It is found that the ANN-based PIs widen considerably when extrapolating and remain constant, or shrink, when interpolating.
• Score: 0.6215404942415159
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Ensuring the reliability and validity of data-driven quadrotor model predictions is essential for their accepted and practical use. This is especially true for grey- and black-box models wherein the mapping of inputs to predictions is not transparent and subsequent reliability notoriously difficult to ascertain. Nonetheless, such techniques are frequently and successfully used to identify quadrotor models. Prediction intervals (PIs) may be employed to provide insight into the consistency and accuracy of model predictions. This paper estimates such PIs for polynomial and Artificial Neural Network (ANN) quadrotor aerodynamic models. Two existing ANN PI estimation techniques - the bootstrap method and the quality driven method - are validated numerically for quadrotor aerodynamic models using an existing high-fidelity quadrotor simulation. Quadrotor aerodynamic models are then identified on real quadrotor flight data to demonstrate their utility and explore their sensitivity to model interpolation and extrapolation. It is found that the ANN-based PIs widen considerably when extrapolating and remain constant, or shrink, when interpolating. While this behaviour also occurs for the polynomial PIs, it is of lower magnitude. The estimated PIs establish probabilistic bounds within which the quadrotor model outputs will likely lie, subject to modelling and measurement uncertainties that are reflected through the PI widths.

Related papers

• Causal Deciphering and Inpainting in Spatio-Temporal Dynamics via Diffusion Model [45.45700202300292]
  CaPaint aims to identify causal regions in data and endow model with causal reasoning ability in a two-stage process. By using a fine-tuned unconditional Diffusion Probabilistic Model (DDPM) as the generative prior, we in-fill the masks defined as environmental parts. Experiments conducted on five real-world ST benchmarks demonstrate that integrating the CaPaint concept allows models to achieve improvements ranging from 4.3% to 77.3%.
  arXiv  Detail & Related papers  (2024-09-29T08:18:50Z)
• Synthesizing Multimodal Electronic Health Records via Predictive Diffusion Models [69.06149482021071]
  We propose a novel EHR data generation model called EHRPD. It is a diffusion-based model designed to predict the next visit based on the current one while also incorporating time interval estimation. We conduct experiments on two public datasets and evaluate EHRPD from fidelity, privacy, and utility perspectives.
  arXiv  Detail & Related papers  (2024-06-20T02:20:23Z)
• GRANP: A Graph Recurrent Attentive Neural Process Model for Vehicle Trajectory Prediction [3.031375888004876]
  We propose a novel model named Graph Recurrent Attentive Neural Process (GRANP) for vehicle trajectory prediction. GRANP contains an encoder with deterministic and latent paths, and a decoder for prediction. We show that GRANP achieves state-of-the-art results and can efficiently quantify uncertainties.
  arXiv  Detail & Related papers  (2024-04-09T05:51:40Z)
• FengWu-4DVar: Coupling the Data-driven Weather Forecasting Model with 4D Variational Assimilation [67.20588721130623]
  We develop an AI-based cyclic weather forecasting system, FengWu-4DVar. FengWu-4DVar can incorporate observational data into the data-driven weather forecasting model. Experiments on the simulated observational dataset demonstrate that FengWu-4DVar is capable of generating reasonable analysis fields.
  arXiv  Detail & Related papers  (2023-12-16T02:07:56Z)
• Context-Aware Generative Models for Prediction of Aircraft Ground Tracks [0.004807514276707785]
  Trajectory prediction plays an important role in supporting the decision-making of Air Traffic Controllers. Traditional TP methods are deterministic and physics-based, with parameters calibrated using aircraft surveillance data harvested across the world. This work proposes a generative method for lateral TP, using probabilistic machine learning to model the effect of the unknown effect of pilot behaviour and ATCO intentions.
  arXiv  Detail & Related papers  (2023-09-26T14:20:09Z)
• EquiDiff: A Conditional Equivariant Diffusion Model For Trajectory Prediction [11.960234424309265]
  We propose EquiDiff, a deep generative model for predicting future vehicle trajectories. EquiDiff is based on the conditional diffusion model, which generates future trajectories by incorporating historical information and random Gaussian noise. Our results demonstrate that EquiDiff outperforms other baseline models in short-term prediction, but has slightly higher errors for long-term prediction.
  arXiv  Detail & Related papers  (2023-08-12T13:17:09Z)
• ChiroDiff: Modelling chirographic data with Diffusion Models [132.5223191478268]
  We introduce a powerful model-class namely "Denoising Diffusion Probabilistic Models" or DDPMs for chirographic data. Our model named "ChiroDiff", being non-autoregressive, learns to capture holistic concepts and therefore remains resilient to higher temporal sampling rate.
  arXiv  Detail & Related papers  (2023-04-07T15:17:48Z)
• Uncovering the Missing Pattern: Unified Framework Towards Trajectory Imputation and Prediction [60.60223171143206]
  Trajectory prediction is a crucial undertaking in understanding entity movement or human behavior from observed sequences. Current methods often assume that the observed sequences are complete while ignoring the potential for missing values. This paper presents a unified framework, the Graph-based Conditional Variational Recurrent Neural Network (GC-VRNN), which can perform trajectory imputation and prediction simultaneously.
  arXiv  Detail & Related papers  (2023-03-28T14:27:27Z)
• Integration of neural network and fuzzy logic decision making compared with bilayered neural network in the simulation of daily dew point temperature [0.8808021343665321]
  dew point temperature (DPT) is simulated using the data-driven approach. Various input patterns, namely T min, T max, and T mean, are utilized for training the architecture.
  arXiv  Detail & Related papers  (2022-02-23T14:25:13Z)
• Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting [98.57851612518758]
  Probabilistic time series forecasting involves estimating the distribution of future based on its history. We propose a deep state space model for probabilistic time series forecasting whereby the non-linear emission model and transition model are parameterized by networks. We show in experiments that our model produces accurate and sharp probabilistic forecasts.
  arXiv  Detail & Related papers  (2021-01-31T06:49:33Z)
• VAE-LIME: Deep Generative Model Based Approach for Local Data-Driven Model Interpretability Applied to the Ironmaking Industry [70.10343492784465]
  It is necessary to expose to the process engineer, not solely the model predictions, but also their interpretability. Model-agnostic local interpretability solutions based on LIME have recently emerged to improve the original method. We present in this paper a novel approach, VAE-LIME, for local interpretability of data-driven models forecasting the temperature of the hot metal produced by a blast furnace.
  arXiv  Detail & Related papers  (2020-07-15T07:07:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.