Neural Network Repair with Reachability Analysis
- URL: http://arxiv.org/abs/2108.04214v1
- Date: Mon, 9 Aug 2021 17:56:51 GMT
- Title: Neural Network Repair with Reachability Analysis
- Authors: Xiaodong Yang, Tom Yamaguchi, Hoang-Dung Tran, Bardh Hoxha, Taylor T
Johnson, Danil Prokhorov
- Abstract summary: Safety is a critical concern for the next generation of autonomy that is likely to rely heavily on deep neural networks for perception and control.
This research proposes a framework to repair unsafe DNNs in safety-critical systems with reachability analysis.
- Score: 10.384532888747993
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Safety is a critical concern for the next generation of autonomy that is
likely to rely heavily on deep neural networks for perception and control.
Formally verifying the safety and robustness of well-trained DNNs and
learning-enabled systems under attacks, model uncertainties, and sensing errors
is essential for safe autonomy. This research proposes a framework to repair
unsafe DNNs in safety-critical systems with reachability analysis. The repair
process is inspired by adversarial training which has demonstrated high
effectiveness in improving the safety and robustness of DNNs. Different from
traditional adversarial training approaches where adversarial examples are
utilized from random attacks and may not be representative of all unsafe
behaviors, our repair process uses reachability analysis to compute the exact
unsafe regions and identify sufficiently representative examples to enhance the
efficacy and efficiency of the adversarial training.
The performance of our framework is evaluated on two types of benchmarks
without safe models as references. One is a DNN controller for aircraft
collision avoidance with access to training data. The other is a rocket lander
where our framework can be seamlessly integrated with the well-known deep
deterministic policy gradient (DDPG) reinforcement learning algorithm. The
experimental results show that our framework can successfully repair all
instances on multiple safety specifications with negligible performance
degradation. In addition, to increase the computational and memory efficiency
of the reachability analysis algorithm, we propose a depth-first-search
algorithm that combines an existing exact analysis method with an
over-approximation approach based on a new set representation. Experimental
results show that our method achieves a five-fold improvement in runtime and a
two-fold improvement in memory usage compared to exact analysis.
Related papers
- Data-Driven Lipschitz Continuity: A Cost-Effective Approach to Improve Adversarial Robustness [47.9744734181236]
We explore the concept of Lipschitz continuity to certify the robustness of deep neural networks (DNNs) against adversarial attacks.
We propose a novel algorithm that remaps the input domain into a constrained range, reducing the Lipschitz constant and potentially enhancing robustness.
Our method achieves the best robust accuracy for CIFAR10, CIFAR100, and ImageNet datasets on the RobustBench leaderboard.
arXiv Detail & Related papers (2024-06-28T03:10:36Z) - Verification of Neural Reachable Tubes via Scenario Optimization and Conformal Prediction [10.40899456282141]
Hamilton-Jacobi reachability analysis is a popular formal verification tool for providing such guarantees.
DeepReach has been used to synthesize reachable tubes and safety controllers for high-dimensional systems.
We propose two verification methods, based on robust scenario optimization and conformal prediction, to provide probabilistic safety guarantees.
arXiv Detail & Related papers (2023-12-14T02:03:36Z) - Doubly Robust Instance-Reweighted Adversarial Training [107.40683655362285]
We propose a novel doubly-robust instance reweighted adversarial framework.
Our importance weights are obtained by optimizing the KL-divergence regularized loss function.
Our proposed approach outperforms related state-of-the-art baseline methods in terms of average robust performance.
arXiv Detail & Related papers (2023-08-01T06:16:18Z) - Robust Feature Inference: A Test-time Defense Strategy using Spectral Projections [12.807619042576018]
We propose a novel test-time defense strategy called Robust Feature Inference (RFI)
RFI is easy to integrate with any existing (robust) training procedure without additional test-time computation.
We show that RFI improves robustness across adaptive and transfer attacks consistently.
arXiv Detail & Related papers (2023-07-21T16:18:58Z) - Evaluating Model-free Reinforcement Learning toward Safety-critical
Tasks [70.76757529955577]
This paper revisits prior work in this scope from the perspective of state-wise safe RL.
We propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection.
To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit.
arXiv Detail & Related papers (2022-12-12T06:30:17Z) - Decorrelative Network Architecture for Robust Electrocardiogram
Classification [4.808817930937323]
It is not possible to train networks that are accurate in all scenarios.
Deep learning methods sample the model parameter space to estimate uncertainty.
These parameters are often subject to the same vulnerabilities, which can be exploited by adversarial attacks.
We propose a novel ensemble approach based on feature decorrelation and Fourier partitioning for teaching networks diverse complementary features.
arXiv Detail & Related papers (2022-07-19T02:36:36Z) - Federated Learning with Unreliable Clients: Performance Analysis and
Mechanism Design [76.29738151117583]
Federated Learning (FL) has become a promising tool for training effective machine learning models among distributed clients.
However, low quality models could be uploaded to the aggregator server by unreliable clients, leading to a degradation or even a collapse of training.
We model these unreliable behaviors of clients and propose a defensive mechanism to mitigate such a security risk.
arXiv Detail & Related papers (2021-05-10T08:02:27Z) - Increasing the Confidence of Deep Neural Networks by Coverage Analysis [71.57324258813674]
This paper presents a lightweight monitoring architecture based on coverage paradigms to enhance the model against different unsafe inputs.
Experimental results show that the proposed approach is effective in detecting both powerful adversarial examples and out-of-distribution inputs.
arXiv Detail & Related papers (2021-01-28T16:38:26Z) - Evaluating the Safety of Deep Reinforcement Learning Models using
Semi-Formal Verification [81.32981236437395]
We present a semi-formal verification approach for decision-making tasks based on interval analysis.
Our method obtains comparable results over standard benchmarks with respect to formal verifiers.
Our approach allows to efficiently evaluate safety properties for decision-making models in practical applications.
arXiv Detail & Related papers (2020-10-19T11:18:06Z) - Opportunities and Challenges in Deep Learning Adversarial Robustness: A
Survey [1.8782750537161614]
This paper studies strategies to implement adversary robustly trained algorithms towards guaranteeing safety in machine learning algorithms.
We provide a taxonomy to classify adversarial attacks and defenses, formulate the Robust Optimization problem in a min-max setting, and divide it into 3 subcategories, namely: Adversarial (re)Training, Regularization Approach, and Certified Defenses.
arXiv Detail & Related papers (2020-07-01T21:00:32Z) - Chance-Constrained Trajectory Optimization for Safe Exploration and
Learning of Nonlinear Systems [81.7983463275447]
Learning-based control algorithms require data collection with abundant supervision for training.
We present a new approach for optimal motion planning with safe exploration that integrates chance-constrained optimal control with dynamics learning and feedback control.
arXiv Detail & Related papers (2020-05-09T05:57:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.