Two Heads are Better than One: Towards Better Adversarial Robustness by Combining Transduction and Rejection

• URL: http://arxiv.org/abs/2305.17528v1
• Date: Sat, 27 May 2023 17:06:17 GMT
• Title: Two Heads are Better than One: Towards Better Adversarial Robustness by Combining Transduction and Rejection
• Abstract summary: We show that a novel application of Tramer's classifier-to-detector technique in the transductive setting can give significantly improved sample-complexity for robust generalization. While our theoretical construction is computationally inefficient, it guides us to identify an efficient transductive algorithm to learn a selective model.
• Score: 34.0779912244438
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Both transduction and rejection have emerged as important techniques for defending against adversarial perturbations. A recent work by Tram\`er showed that, in the rejection-only case (no transduction), a strong rejection-solution can be turned into a strong (but computationally inefficient) non-rejection solution. This detector-to-classifier reduction has been mostly applied to give evidence that certain claims of strong selective-model solutions are susceptible, leaving the benefits of rejection unclear. On the other hand, a recent work by Goldwasser et al. showed that rejection combined with transduction can give provable guarantees (for certain problems) that cannot be achieved otherwise. Nevertheless, under recent strong adversarial attacks (GMSA, which has been shown to be much more effective than AutoAttack against transduction), Goldwasser et al.'s work was shown to have low performance in a practical deep-learning setting. In this paper, we take a step towards realizing the promise of transduction+rejection in more realistic scenarios. Theoretically, we show that a novel application of Tram\`er's classifier-to-detector technique in the transductive setting can give significantly improved sample-complexity for robust generalization. While our theoretical construction is computationally inefficient, it guides us to identify an efficient transductive algorithm to learn a selective model. Extensive experiments using state of the art attacks (AutoAttack, GMSA) show that our solutions provide significantly better robust accuracy.

Related papers

• Improving Adversarial Robustness via Decoupled Visual Representation Masking [65.73203518658224]
  In this paper, we highlight two novel properties of robust features from the feature distribution perspective. We find that state-of-the-art defense methods aim to address both of these mentioned issues well. Specifically, we propose a simple but effective defense based on decoupled visual representation masking.
  arXiv  Detail & Related papers  (2024-06-16T13:29:41Z)
• Purify Unlearnable Examples via Rate-Constrained Variational Autoencoders [101.42201747763178]
  Unlearnable examples (UEs) seek to maximize testing error by making subtle modifications to training examples that are correctly labeled. Our work provides a novel disentanglement mechanism to build an efficient pre-training purification method.
  arXiv  Detail & Related papers  (2024-05-02T16:49:25Z)
• Shortcuts Arising from Contrast: Effective and Covert Clean-Label Attacks in Prompt-Based Learning [40.130762098868736]
  We propose a method named Contrastive Shortcut Injection (CSI), by leveraging activation values, integrates trigger design and data selection strategies to craft stronger shortcut features. With extensive experiments on full-shot and few-shot text classification tasks, we empirically validate CSI's high effectiveness and high stealthiness at low poisoning rates.
  arXiv  Detail & Related papers  (2024-03-30T20:02:36Z)
• Adversarial Training on Purification (AToP): Advancing Both Robustness and Generalization [29.09894840783714]
  We propose a novel pipeline to acquire the robust purifier model, named Adversarial Training on Purification (AToP) To evaluate our method in an efficient and scalable way, we conduct extensive experiments on CIFAR-10, CIFAR-100, and ImageNette.
  arXiv  Detail & Related papers  (2024-01-29T17:56:42Z)
• Perturbation-Invariant Adversarial Training for Neural Ranking Models: Improving the Effectiveness-Robustness Trade-Off [107.35833747750446]
  adversarial examples can be crafted by adding imperceptible perturbations to legitimate documents. This vulnerability raises significant concerns about their reliability and hinders the widespread deployment of NRMs. In this study, we establish theoretical guarantees regarding the effectiveness-robustness trade-off in NRMs.
  arXiv  Detail & Related papers  (2023-12-16T05:38:39Z)
• Closing the Gap: Achieving Better Accuracy-Robustness Tradeoffs against Query-Based Attacks [1.54994260281059]
  We show how to efficiently establish, at test-time, a solid tradeoff between robustness and accuracy when mitigating query-based attacks. Our approach is independent of training and supported by theory.
  arXiv  Detail & Related papers  (2023-12-15T17:02:19Z)
• DiffAttack: Evasion Attacks Against Diffusion-Based Adversarial Purification [63.65630243675792]
  Diffusion-based purification defenses leverage diffusion models to remove crafted perturbations of adversarial examples. Recent studies show that even advanced attacks cannot break such defenses effectively. We propose a unified framework DiffAttack to perform effective and efficient attacks against diffusion-based purification defenses.
  arXiv  Detail & Related papers  (2023-10-27T15:17:50Z)
• Stratified Adversarial Robustness with Rejection [33.72077702550626]
  We study adversarially-robust classification with rejection in the stratified rejection setting. We propose a novel defense method -- Adversarial Training with Consistent Prediction-based Rejection (CPR) CPR significantly outperforms existing methods under strong adaptive attacks.
  arXiv  Detail & Related papers  (2023-05-02T01:04:29Z)
• Logit Margin Matters: Improving Transferable Targeted Adversarial Attack by Logit Calibration [85.71545080119026]
  Cross-Entropy (CE) loss function is insufficient to learn transferable targeted adversarial examples. We propose two simple and effective logit calibration methods, which are achieved by downscaling the logits with a temperature factor and an adaptive margin. Experiments conducted on the ImageNet dataset validate the effectiveness of the proposed methods.
  arXiv  Detail & Related papers  (2023-03-07T06:42:52Z)
• A Large-scale Multiple-objective Method for Black-box Attack against Object Detection [70.00150794625053]
  We propose to minimize the true positive rate and maximize the false positive rate, which can encourage more false positive objects to block the generation of new true positive bounding boxes. We extend the standard Genetic Algorithm with Random Subset selection and Divide-and-Conquer, called GARSDC, which significantly improves the efficiency. Compared with the state-of-art attack methods, GARSDC decreases by an average 12.0 in the mAP and queries by about 1000 times in extensive experiments.
  arXiv  Detail & Related papers  (2022-09-16T08:36:42Z)
• Two Heads are Better than One: Robust Learning Meets Multi-branch Models [9.748686016224102]
  Adversarial training, a reliable and effective method of defense, may significantly reduce the vulnerability of neural networks.<n>We propose textitBranch Orthogonality adveRsarial Training (BORT) to obtain state-of-the-art performance with solely the original dataset for adversarial training.<n>We evaluate our approach on CIFAR-10, CIFAR-100 and SVHN against $ell_infty$ norm-bounded perturbations of size $epsilon = 8/255$, respectively.
  arXiv  Detail & Related papers  (2022-08-17T05:42:59Z)
• Few-shot Forgery Detection via Guided Adversarial Interpolation [56.59499187594308]
  Existing forgery detection methods suffer from significant performance drops when applied to unseen novel forgery approaches. We propose Guided Adversarial Interpolation (GAI) to overcome the few-shot forgery detection problem. Our method is validated to be robust to choices of majority and minority forgery approaches.
  arXiv  Detail & Related papers  (2022-04-12T16:05:10Z)
• ROPUST: Improving Robustness through Fine-tuning with Photonic Processors and Synthetic Gradients [65.52888259961803]
  We introduce ROPUST, a simple and efficient method to leverage robust pre-trained models and increase their robustness. We test our method on nine different models against four attacks in RobustBench, consistently improving over state-of-the-art performance. We show that even with state-of-the-art phase retrieval techniques, ROPUST remains an effective defense.
  arXiv  Detail & Related papers  (2021-07-06T12:03:36Z)
• Adaptive Feature Alignment for Adversarial Training [56.17654691470554]
  CNNs are typically vulnerable to adversarial attacks, which pose a threat to security-sensitive applications. We propose the adaptive feature alignment (AFA) to generate features of arbitrary attacking strengths. Our method is trained to automatically align features of arbitrary attacking strength.
  arXiv  Detail & Related papers  (2021-05-31T17:01:05Z)
• Going Far Boosts Attack Transferability, but Do Not Do It [16.901240544106948]
  We investigate the impacts of optimization on attack transferability by comprehensive experiments concerning 7 optimization algorithms, 4 surrogates, and 9 black-box models. We surprisingly find that the varied transferability of AEs from optimization algorithms is strongly related to the Root Mean Square Error (RMSE) from their original samples. Although LARA significantly improves transferability by 20%, it is insufficient to exploit the vulnerability of DNNs.
  arXiv  Detail & Related papers  (2021-02-20T13:19:31Z)
• ATRO: Adversarial Training with a Rejection Option [10.36668157679368]
  This paper proposes a classification framework with a rejection option to mitigate the performance deterioration caused by adversarial examples. Applying the adversarial training objective to both a classifier and a rejection function simultaneously, we can choose to abstain from classification when it has insufficient confidence to classify a test data point.
  arXiv  Detail & Related papers  (2020-10-24T14:05:03Z)
• Global Optimization of Objective Functions Represented by ReLU Networks [77.55969359556032]
  Neural networks can learn complex, non- adversarial functions, and it is challenging to guarantee their correct behavior in safety-critical contexts. Many approaches exist to find failures in networks (e.g., adversarial examples), but these cannot guarantee the absence of failures. We propose an approach that integrates the optimization process into the verification procedure, achieving better performance than the naive approach.
  arXiv  Detail & Related papers  (2020-10-07T08:19:48Z)
• Bag of Tricks for Adversarial Training [50.53525358778331]
  Adrial training is one of the most effective strategies for promoting model robustness. Recent benchmarks show that most of the proposed improvements on AT are less effective than simply early stopping the training procedure.
  arXiv  Detail & Related papers  (2020-10-01T15:03:51Z)
• Adversarial robustness via stochastic regularization of neural activation sensitivity [24.02105949163359]
  We suggest a novel defense mechanism that simultaneously addresses both defense goals. We flatten the gradients of the loss surface, making adversarial examples harder to find. In addition, we push the decision away from correctly classified inputs by leveraging Jacobian regularization.
  arXiv  Detail & Related papers  (2020-09-23T19:31:55Z)
• TEAM: We Need More Powerful Adversarial Examples for DNNs [6.7943676146532885]
  Adversarial examples can lead to misclassification of deep neural networks (DNNs) We propose a novel method to generate more powerful adversarial examples than previous methods. Our method can reliably produce adversarial examples with 100% attack success rate (ASR) while only by smaller perturbations.
  arXiv  Detail & Related papers  (2020-07-31T04:11:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.