PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures
- URL: http://arxiv.org/abs/2112.05135v2
- Date: Sat, 11 Dec 2021 19:19:18 GMT
- Title: PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures
- Authors: Dan Hendrycks and Andy Zou and Mantas Mazeika and Leonard Tang and Bo
Li and Dawn Song and Jacob Steinhardt
- Abstract summary: We propose a new data augmentation strategy utilizing the natural structural complexity of pictures such as fractals.
To meet this challenge, we design a new data augmentation strategy utilizing the natural structural complexity of pictures such as fractals.
- Score: 65.36234499099294
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In real-world applications of machine learning, reliable and safe systems
must consider measures of performance beyond standard test set accuracy. These
other goals include out-of-distribution (OOD) robustness, prediction
consistency, resilience to adversaries, calibrated uncertainty estimates, and
the ability to detect anomalous inputs. However, improving performance towards
these goals is often a balancing act that today's methods cannot achieve
without sacrificing performance on other safety axes. For instance, adversarial
training improves adversarial robustness but sharply degrades other classifier
performance metrics. Similarly, strong data augmentation and regularization
techniques often improve OOD robustness but harm anomaly detection, raising the
question of whether a Pareto improvement on all existing safety measures is
possible. To meet this challenge, we design a new data augmentation strategy
utilizing the natural structural complexity of pictures such as fractals, which
outperforms numerous baselines, is near Pareto-optimal, and roundly improves
safety measures.
Related papers
- Data-Driven Lipschitz Continuity: A Cost-Effective Approach to Improve Adversarial Robustness [47.9744734181236]
We explore the concept of Lipschitz continuity to certify the robustness of deep neural networks (DNNs) against adversarial attacks.
We propose a novel algorithm that remaps the input domain into a constrained range, reducing the Lipschitz constant and potentially enhancing robustness.
Our method achieves the best robust accuracy for CIFAR10, CIFAR100, and ImageNet datasets on the RobustBench leaderboard.
arXiv Detail & Related papers (2024-06-28T03:10:36Z) - Towards Precise Observations of Neural Model Robustness in Classification [2.127049691404299]
In deep learning applications, robustness measures the ability of neural models that handle slight changes in input data.
Our approach contributes to a deeper understanding of model robustness in safety-critical applications.
arXiv Detail & Related papers (2024-04-25T09:37:44Z) - Exploring the Adversarial Frontier: Quantifying Robustness via
Adversarial Hypervolume [18.4516572499628]
We propose a new metric termed adversarial hypervolume, assessing the robustness of deep learning models comprehensively over a range of perturbation intensities.
We adopt a novel training algorithm that enhances adversarial robustness uniformly across various perturbation intensities.
This research contributes a new measure of robustness and establishes a standard for assessing benchmarking and the resilience of current and future defensive models against adversarial threats.
arXiv Detail & Related papers (2024-03-08T07:03:18Z) - FLIP: A Provable Defense Framework for Backdoor Mitigation in Federated
Learning [66.56240101249803]
We study how hardening benign clients can affect the global model (and the malicious clients)
We propose a trigger reverse engineering based defense and show that our method can achieve improvement with guarantee robustness.
Our results on eight competing SOTA defense methods show the empirical superiority of our method on both single-shot and continuous FL backdoor attacks.
arXiv Detail & Related papers (2022-10-23T22:24:03Z) - Decorrelative Network Architecture for Robust Electrocardiogram
Classification [4.808817930937323]
It is not possible to train networks that are accurate in all scenarios.
Deep learning methods sample the model parameter space to estimate uncertainty.
These parameters are often subject to the same vulnerabilities, which can be exploited by adversarial attacks.
We propose a novel ensemble approach based on feature decorrelation and Fourier partitioning for teaching networks diverse complementary features.
arXiv Detail & Related papers (2022-07-19T02:36:36Z) - How many perturbations break this model? Evaluating robustness beyond
adversarial accuracy [28.934863462633636]
We introduce adversarial sparsity, which quantifies how difficult it is to find a successful perturbation given both an input point and a constraint on the direction of the perturbation.
We show that sparsity provides valuable insight into neural networks in multiple ways.
arXiv Detail & Related papers (2022-07-08T21:25:17Z) - On the Robustness of Quality Measures for GANs [136.18799984346248]
This work evaluates the robustness of quality measures of generative models such as Inception Score (IS) and Fr'echet Inception Distance (FID)
We show that such metrics can also be manipulated by additive pixel perturbations.
arXiv Detail & Related papers (2022-01-31T06:43:09Z) - Certified Adversarial Defenses Meet Out-of-Distribution Corruptions:
Benchmarking Robustness and Simple Baselines [65.0803400763215]
This work critically examines how adversarial robustness guarantees change when state-of-the-art certifiably robust models encounter out-of-distribution data.
We propose a novel data augmentation scheme, FourierMix, that produces augmentations to improve the spectral coverage of the training data.
We find that FourierMix augmentations help eliminate the spectral bias of certifiably robust models enabling them to achieve significantly better robustness guarantees on a range of OOD benchmarks.
arXiv Detail & Related papers (2021-12-01T17:11:22Z) - Neural Network Repair with Reachability Analysis [10.384532888747993]
Safety is a critical concern for the next generation of autonomy that is likely to rely heavily on deep neural networks for perception and control.
This research proposes a framework to repair unsafe DNNs in safety-critical systems with reachability analysis.
arXiv Detail & Related papers (2021-08-09T17:56:51Z) - Trust but Verify: Assigning Prediction Credibility by Counterfactual
Constrained Learning [123.3472310767721]
Prediction credibility measures are fundamental in statistics and machine learning.
These measures should account for the wide variety of models used in practice.
The framework developed in this work expresses the credibility as a risk-fit trade-off.
arXiv Detail & Related papers (2020-11-24T19:52:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.