Validation of Composite Systems by Discrepancy Propagation
- URL: http://arxiv.org/abs/2210.12061v2
- Date: Wed, 3 Jan 2024 16:10:50 GMT
- Title: Validation of Composite Systems by Discrepancy Propagation
- Authors: David Reeb, Kanil Patel, Karim Barsim, Martin Schiegg, Sebastian
Gerwinn
- Abstract summary: We present a validation method that propagates bounds on distributional discrepancy measures through a composite system.
We demonstrate that our propagation method yields valid and useful bounds for composite systems exhibiting a variety of realistic effects.
- Score: 4.588222946914529
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Assessing the validity of a real-world system with respect to given quality
criteria is a common yet costly task in industrial applications due to the vast
number of required real-world tests. Validating such systems by means of
simulation offers a promising and less expensive alternative, but requires an
assessment of the simulation accuracy and therefore end-to-end measurements.
Additionally, covariate shifts between simulations and actual usage can cause
difficulties for estimating the reliability of such systems. In this work, we
present a validation method that propagates bounds on distributional
discrepancy measures through a composite system, thereby allowing us to derive
an upper bound on the failure probability of the real system from potentially
inaccurate simulations. Each propagation step entails an optimization problem,
where -- for measures such as maximum mean discrepancy (MMD) -- we develop
tight convex relaxations based on semidefinite programs. We demonstrate that
our propagation method yields valid and useful bounds for composite systems
exhibiting a variety of realistic effects. In particular, we show that the
proposed method can successfully account for data shifts within the
experimental design as well as model inaccuracies within the simulation.
Related papers
- Addressing Misspecification in Simulation-based Inference through Data-driven Calibration [43.811367860375825]
Recent work has demonstrated that model misspecification can harm simulation-based inference's reliability.
This work introduces robust posterior estimation (ROPE), a framework that overcomes model misspecification with a small real-world calibration set of ground truth parameter measurements.
arXiv Detail & Related papers (2024-05-14T16:04:39Z) - Discovering Decision Manifolds to Assure Trusted Autonomous Systems [0.0]
We propose an optimization-based search technique for capturing the range of correct and incorrect responses a system could exhibit.
This manifold provides a more detailed understanding of system reliability than traditional testing or Monte Carlo simulations.
In this proof-of-concept, we apply our method to a software-in-the-loop evaluation of an autonomous vehicle.
arXiv Detail & Related papers (2024-02-12T16:55:58Z) - Distributionally Robust Model-based Reinforcement Learning with Large
State Spaces [55.14361269378122]
Three major challenges in reinforcement learning are the complex dynamical systems with large state spaces, the costly data acquisition processes, and the deviation of real-world dynamics from the training environment deployment.
We study distributionally robust Markov decision processes with continuous state spaces under the widely used Kullback-Leibler, chi-square, and total variation uncertainty sets.
We propose a model-based approach that utilizes Gaussian Processes and the maximum variance reduction algorithm to efficiently learn multi-output nominal transition dynamics.
arXiv Detail & Related papers (2023-09-05T13:42:11Z) - Learning-Based Optimal Control with Performance Guarantees for Unknown Systems with Latent States [4.4820711784498]
This paper proposes a novel method for the computation of an optimal input trajectory for unknown nonlinear systems with latent states.
The effectiveness of the proposed method is demonstrated in a numerical simulation.
arXiv Detail & Related papers (2023-03-31T11:06:09Z) - Robust Control for Dynamical Systems With Non-Gaussian Noise via Formal
Abstractions [59.605246463200736]
We present a novel controller synthesis method that does not rely on any explicit representation of the noise distributions.
First, we abstract the continuous control system into a finite-state model that captures noise by probabilistic transitions between discrete states.
We use state-of-the-art verification techniques to provide guarantees on the interval Markov decision process and compute a controller for which these guarantees carry over to the original control system.
arXiv Detail & Related papers (2023-01-04T10:40:30Z) - Importance sampling for stochastic quantum simulations [68.8204255655161]
We introduce the qDrift protocol, which builds random product formulas by sampling from the Hamiltonian according to the coefficients.
We show that the simulation cost can be reduced while achieving the same accuracy, by considering the individual simulation cost during the sampling stage.
Results are confirmed by numerical simulations performed on a lattice nuclear effective field theory.
arXiv Detail & Related papers (2022-12-12T15:06:32Z) - Testing Rare Downstream Safety Violations via Upstream Adaptive Sampling
of Perception Error Models [20.815131169609316]
This paper combines perception error models -- surrogates for a sensor-based detection system -- with state-dependent adaptive importance sampling.
Experiments with an autonomous braking system equipped with an RGB obstacle-detector show that our method can calculate accurate failure probabilities.
arXiv Detail & Related papers (2022-09-20T12:26:06Z) - Probabilistic robust linear quadratic regulators with Gaussian processes [73.0364959221845]
Probabilistic models such as Gaussian processes (GPs) are powerful tools to learn unknown dynamical systems from data for subsequent use in control design.
We present a novel controller synthesis for linearized GP dynamics that yields robust controllers with respect to a probabilistic stability margin.
arXiv Detail & Related papers (2021-05-17T08:36:18Z) - GenDICE: Generalized Offline Estimation of Stationary Values [108.17309783125398]
We show that effective estimation can still be achieved in important applications.
Our approach is based on estimating a ratio that corrects for the discrepancy between the stationary and empirical distributions.
The resulting algorithm, GenDICE, is straightforward and effective.
arXiv Detail & Related papers (2020-02-21T00:27:52Z) - DISCO: Double Likelihood-free Inference Stochastic Control [29.84276469617019]
We propose to leverage the power of modern simulators and recent techniques in Bayesian statistics for likelihood-free inference.
The posterior distribution over simulation parameters is propagated through a potentially non-analytical model of the system.
Experiments show that the controller proposed attained superior performance and robustness on classical control and robotics tasks.
arXiv Detail & Related papers (2020-02-18T05:29:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.