Validation of Composite Systems by Discrepancy Propagation
- URL: http://arxiv.org/abs/2210.12061v2
- Date: Wed, 3 Jan 2024 16:10:50 GMT
- Title: Validation of Composite Systems by Discrepancy Propagation
- Authors: David Reeb, Kanil Patel, Karim Barsim, Martin Schiegg, Sebastian
Gerwinn
- Abstract summary: We present a validation method that propagates bounds on distributional discrepancy measures through a composite system.
We demonstrate that our propagation method yields valid and useful bounds for composite systems exhibiting a variety of realistic effects.
- Score: 4.588222946914529
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Assessing the validity of a real-world system with respect to given quality
criteria is a common yet costly task in industrial applications due to the vast
number of required real-world tests. Validating such systems by means of
simulation offers a promising and less expensive alternative, but requires an
assessment of the simulation accuracy and therefore end-to-end measurements.
Additionally, covariate shifts between simulations and actual usage can cause
difficulties for estimating the reliability of such systems. In this work, we
present a validation method that propagates bounds on distributional
discrepancy measures through a composite system, thereby allowing us to derive
an upper bound on the failure probability of the real system from potentially
inaccurate simulations. Each propagation step entails an optimization problem,
where -- for measures such as maximum mean discrepancy (MMD) -- we develop
tight convex relaxations based on semidefinite programs. We demonstrate that
our propagation method yields valid and useful bounds for composite systems
exhibiting a variety of realistic effects. In particular, we show that the
proposed method can successfully account for data shifts within the
experimental design as well as model inaccuracies within the simulation.
Related papers
- Learning Controlled Stochastic Differential Equations [61.82896036131116]
This work proposes a novel method for estimating both drift and diffusion coefficients of continuous, multidimensional, nonlinear controlled differential equations with non-uniform diffusion.
We provide strong theoretical guarantees, including finite-sample bounds for (L2), (Linfty), and risk metrics, with learning rates adaptive to coefficients' regularity.
Our method is available as an open-source Python library.
arXiv Detail & Related papers (2024-11-04T11:09:58Z) - Exogenous Matching: Learning Good Proposals for Tractable Counterfactual Estimation [1.9662978733004601]
We propose an importance sampling method for tractable and efficient estimation of counterfactual expressions.
By minimizing a common upper bound of counterfactual estimators, we transform the variance minimization problem into a conditional distribution learning problem.
We validate the theoretical results through experiments under various types and settings of Structural Causal Models (SCMs) and demonstrate the outperformance on counterfactual estimation tasks.
arXiv Detail & Related papers (2024-10-17T03:08:28Z) - Addressing Misspecification in Simulation-based Inference through Data-driven Calibration [43.811367860375825]
Recent work has demonstrated that model misspecification can harm simulation-based inference's reliability.
This work introduces robust posterior estimation (ROPE), a framework that overcomes model misspecification with a small real-world calibration set of ground truth parameter measurements.
arXiv Detail & Related papers (2024-05-14T16:04:39Z) - Distributionally Robust Model-based Reinforcement Learning with Large
State Spaces [55.14361269378122]
Three major challenges in reinforcement learning are the complex dynamical systems with large state spaces, the costly data acquisition processes, and the deviation of real-world dynamics from the training environment deployment.
We study distributionally robust Markov decision processes with continuous state spaces under the widely used Kullback-Leibler, chi-square, and total variation uncertainty sets.
We propose a model-based approach that utilizes Gaussian Processes and the maximum variance reduction algorithm to efficiently learn multi-output nominal transition dynamics.
arXiv Detail & Related papers (2023-09-05T13:42:11Z) - Learning-Based Optimal Control with Performance Guarantees for Unknown Systems with Latent States [4.4820711784498]
This paper proposes a novel method for the computation of an optimal input trajectory for unknown nonlinear systems with latent states.
The effectiveness of the proposed method is demonstrated in a numerical simulation.
arXiv Detail & Related papers (2023-03-31T11:06:09Z) - Importance sampling for stochastic quantum simulations [68.8204255655161]
We introduce the qDrift protocol, which builds random product formulas by sampling from the Hamiltonian according to the coefficients.
We show that the simulation cost can be reduced while achieving the same accuracy, by considering the individual simulation cost during the sampling stage.
Results are confirmed by numerical simulations performed on a lattice nuclear effective field theory.
arXiv Detail & Related papers (2022-12-12T15:06:32Z) - Testing Rare Downstream Safety Violations via Upstream Adaptive Sampling
of Perception Error Models [20.815131169609316]
This paper combines perception error models -- surrogates for a sensor-based detection system -- with state-dependent adaptive importance sampling.
Experiments with an autonomous braking system equipped with an RGB obstacle-detector show that our method can calculate accurate failure probabilities.
arXiv Detail & Related papers (2022-09-20T12:26:06Z) - Probabilistic robust linear quadratic regulators with Gaussian processes [73.0364959221845]
Probabilistic models such as Gaussian processes (GPs) are powerful tools to learn unknown dynamical systems from data for subsequent use in control design.
We present a novel controller synthesis for linearized GP dynamics that yields robust controllers with respect to a probabilistic stability margin.
arXiv Detail & Related papers (2021-05-17T08:36:18Z) - GenDICE: Generalized Offline Estimation of Stationary Values [108.17309783125398]
We show that effective estimation can still be achieved in important applications.
Our approach is based on estimating a ratio that corrects for the discrepancy between the stationary and empirical distributions.
The resulting algorithm, GenDICE, is straightforward and effective.
arXiv Detail & Related papers (2020-02-21T00:27:52Z) - DISCO: Double Likelihood-free Inference Stochastic Control [29.84276469617019]
We propose to leverage the power of modern simulators and recent techniques in Bayesian statistics for likelihood-free inference.
The posterior distribution over simulation parameters is propagated through a potentially non-analytical model of the system.
Experiments show that the controller proposed attained superior performance and robustness on classical control and robotics tasks.
arXiv Detail & Related papers (2020-02-18T05:29:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.