ODG-Q: Robust Quantization via Online Domain Generalization
- URL: http://arxiv.org/abs/2210.08701v1
- Date: Mon, 17 Oct 2022 02:25:28 GMT
- Title: ODG-Q: Robust Quantization via Online Domain Generalization
- Authors: Chaofan Tao, Ngai Wong
- Abstract summary: Quantizing neural networks to low-bitwidth is important for model deployment on resource-limited edge hardware.
We propose a new method by recasting robust quantization as an online domain generalization problem, termed ODG-Q.
ODG-Q consistently outperforms existing works against various adversarial attacks.
- Score: 9.25177374431812
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Quantizing neural networks to low-bitwidth is important for model deployment
on resource-limited edge hardware. Although a quantized network has a smaller
model size and memory footprint, it is fragile to adversarial attacks. However,
few methods study the robustness and training efficiency of quantized networks.
To this end, we propose a new method by recasting robust quantization as an
online domain generalization problem, termed ODG-Q, which generates diverse
adversarial data at a low cost during training. ODG-Q consistently outperforms
existing works against various adversarial attacks. For example, on CIFAR-10
dataset, ODG-Q achieves 49.2% average improvements under five common white-box
attacks and 21.7% average improvements under five common black-box attacks,
with a training cost similar to that of natural training (viz. without
adversaries). To our best knowledge, this work is the first work that trains
both quantized and binary neural networks on ImageNet that consistently improve
robustness under different attacks. We also provide a theoretical insight of
ODG-Q that accounts for the bound of model risk on attacked data.
Related papers
- Improving the Robustness of Quantized Deep Neural Networks to White-Box
Attacks using Stochastic Quantization and Information-Theoretic Ensemble
Training [1.6098666134798774]
Most real-world applications that employ deep neural networks (DNNs) quantize them to low precision to reduce the compute needs.
We present a method to improve the robustness of quantized DNNs to white-box adversarial attacks.
arXiv Detail & Related papers (2023-11-30T17:15:58Z) - Everything Perturbed All at Once: Enabling Differentiable Graph Attacks [61.61327182050706]
Graph neural networks (GNNs) have been shown to be vulnerable to adversarial attacks.
We propose a novel attack method called Differentiable Graph Attack (DGA) to efficiently generate effective attacks.
Compared to the state-of-the-art, DGA achieves nearly equivalent attack performance with 6 times less training time and 11 times smaller GPU memory footprint.
arXiv Detail & Related papers (2023-08-29T20:14:42Z) - Quantization Aware Attack: Enhancing Transferable Adversarial Attacks by Model Quantization [57.87950229651958]
Quantized neural networks (QNNs) have received increasing attention in resource-constrained scenarios due to their exceptional generalizability.
Previous studies claim that transferability is difficult to achieve across QNNs with different bitwidths.
We propose textitquantization aware attack (QAA) which fine-tunes a QNN substitute model with a multiple-bitwidth training objective.
arXiv Detail & Related papers (2023-05-10T03:46:53Z) - Two Heads are Better than One: Robust Learning Meets Multi-branch Models [14.72099568017039]
We propose Branch Orthogonality adveRsarial Training (BORT) to obtain state-of-the-art performance with solely the original dataset for adversarial training.
We evaluate our approach on CIFAR-10, CIFAR-100, and SVHN against ell_infty norm-bounded perturbations of size epsilon = 8/255, respectively.
arXiv Detail & Related papers (2022-08-17T05:42:59Z) - Distributed Adversarial Training to Robustify Deep Neural Networks at
Scale [100.19539096465101]
Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification.
To defend against such attacks, an effective approach, known as adversarial training (AT), has been shown to mitigate robust training.
We propose a large-batch adversarial training framework implemented over multiple machines.
arXiv Detail & Related papers (2022-06-13T15:39:43Z) - Practical No-box Adversarial Attacks with Training-free Hybrid Image
Transformation [123.33816363589506]
We show the existence of a textbftraining-free adversarial perturbation under the no-box threat model.
Motivated by our observation that high-frequency component (HFC) domains in low-level features, we attack an image mainly by manipulating its frequency components.
Our method is even competitive to mainstream transfer-based black-box attacks.
arXiv Detail & Related papers (2022-03-09T09:51:00Z) - Robustness of Graph Neural Networks at Scale [63.45769413975601]
We study how to attack and defend Graph Neural Networks (GNNs) at scale.
We propose two sparsity-aware first-order optimization attacks that maintain an efficient representation.
We show that common surrogate losses are not well-suited for global attacks on GNNs.
arXiv Detail & Related papers (2021-10-26T21:31:17Z) - Compact representations of convolutional neural networks via weight
pruning and quantization [63.417651529192014]
We propose a novel storage format for convolutional neural networks (CNNs) based on source coding and leveraging both weight pruning and quantization.
We achieve a reduction of space occupancy up to 0.6% on fully connected layers and 5.44% on the whole network, while performing at least as competitive as the baseline.
arXiv Detail & Related papers (2021-08-28T20:39:54Z) - Towards More Practical Adversarial Attacks on Graph Neural Networks [14.78539966828287]
We study the black-box attacks on graph neural networks (GNNs) under a novel and realistic constraint.
We show that the structural inductive biases of GNN models can be an effective source for this type of attacks.
arXiv Detail & Related papers (2020-06-09T05:27:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.