David and Goliath: An Empirical Evaluation of Attacks and Defenses for QNNs at the Deep Edge

• URL: http://arxiv.org/abs/2404.05688v2
• Date: Thu, 2 May 2024 20:09:15 GMT
• Title: David and Goliath: An Empirical Evaluation of Attacks and Defenses for QNNs at the Deep Edge
• Abstract summary: Quantization is a well-established technique that has proved effective in enabling the deployment of neural networks on MCUs. We empirically evaluate the effectiveness of attacks and defenses from (full-precision) ANNs onconstrained QNNs.
• Score: 0.9606184401853288
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: ML is shifting from the cloud to the edge. Edge computing reduces the surface exposing private data and enables reliable throughput guarantees in real-time applications. Of the panoply of devices deployed at the edge, resource-constrained MCUs, e.g., Arm Cortex-M, are more prevalent, orders of magnitude cheaper, and less power-hungry than application processors or GPUs. Thus, enabling intelligence at the deep edge is the zeitgeist, with researchers focusing on unveiling novel approaches to deploy ANNs on these constrained devices. Quantization is a well-established technique that has proved effective in enabling the deployment of neural networks on MCUs; however, it is still an open question to understand the robustness of QNNs in the face of adversarial examples. To fill this gap, we empirically evaluate the effectiveness of attacks and defenses from (full-precision) ANNs on (constrained) QNNs. Our evaluation includes three QNNs targeting TinyML applications, ten attacks, and six defenses. With this study, we draw a set of interesting findings. First, quantization increases the point distance to the decision boundary and leads the gradient estimated by some attacks to explode or vanish. Second, quantization can act as a noise attenuator or amplifier, depending on the noise magnitude, and causes gradient misalignment. Regarding adversarial defenses, we conclude that input pre-processing defenses show impressive results on small perturbations; however, they fall short as the perturbation increases. At the same time, train-based defenses increase the average point distance to the decision boundary, which holds after quantization. However, we argue that train-based defenses still need to smooth the quantization-shift and gradient misalignment phenomenons to counteract adversarial example transferability to QNNs. All artifacts are open-sourced to enable independent validation of results.

Related papers

• Localization then Neutralization: Gradient-guided Token Suppression against Visual Prompt Injection Attack [78.02110947708535]
  Adversarial images pose a severe security threat to multimodal large language models through prompt injection.<n>We show that successful adversarial attacks do not rely on the entire image uniformly but instead depend on a small subset of critical image tokens.<n>We propose Gradient Token Masking (GTM), which localizes these tokens via gradient analysis and neutralizes them through masking.
  arXiv  Detail & Related papers  (2026-05-24T17:51:34Z)
• TriQDef: Disrupting Semantic and Gradient Alignment to Prevent Adversarial Patch Transferability in Quantized Neural Networks [3.962831477787584]
  Quantized Neural Networks (QNNs) are increasingly deployed in edge and resource-constrained environments.<n>TriQDef is a tri-level quantization-aware defense framework designed to disrupt the transferability of patch-based adversarial attacks across QNNs.
  arXiv  Detail & Related papers  (2025-08-16T18:51:48Z)
• Breaking the Limits of Quantization-Aware Defenses: QADT-R for Robustness Against Patch-Based Adversarial Attacks in QNNs [3.962831477787584]
  Quantized Neural Networks (QNNs) have emerged as a promising solution for reducing model size and computational costs. In this work, we demonstrate that adversarial patches remain highly transferable across quantized models. We propose Quantization-Aware Defense Training with Randomization (QADT-R) to enhance resilience against highly transferable patch-based attacks.
  arXiv  Detail & Related papers  (2025-03-10T08:43:36Z)
• Augmented Neural Fine-Tuning for Efficient Backdoor Purification [16.74156528484354]
  Recent studies have revealed the vulnerability of deep neural networks (DNNs) to various backdoor attacks. We propose Neural mask Fine-Tuning (NFT) with an aim to optimally re-organize the neuron activities. NFT relaxes the trigger synthesis process and eliminates the requirement of the adversarial search module.
  arXiv  Detail & Related papers  (2024-07-14T02:36:54Z)
• Rethinking PGD Attack: Is Sign Function Necessary? [131.6894310945647]
  We present a theoretical analysis of how such sign-based update algorithm influences step-wise attack performance. We propose a new raw gradient descent (RGD) algorithm that eliminates the use of sign. The effectiveness of the proposed RGD algorithm has been demonstrated extensively in experiments.
  arXiv  Detail & Related papers  (2023-12-03T02:26:58Z)
• Quantization Aware Attack: Enhancing Transferable Adversarial Attacks by Model Quantization [57.87950229651958]
  Quantized neural networks (QNNs) have received increasing attention in resource-constrained scenarios due to their exceptional generalizability. Previous studies claim that transferability is difficult to achieve across QNNs with different bitwidths. We propose textitquantization aware attack (QAA) which fine-tunes a QNN substitute model with a multiple-bitwidth training objective.
  arXiv  Detail & Related papers  (2023-05-10T03:46:53Z)
• Overload: Latency Attacks on Object Detection for Edge Devices [47.9744734181236]
  This paper investigates latency attacks on deep learning applications. Unlike common adversarial attacks for misclassification, the goal of latency attacks is to increase the inference time. We use object detection to demonstrate how such kind of attacks work.
  arXiv  Detail & Related papers  (2023-04-11T17:24:31Z)
• Qu-ANTI-zation: Exploiting Quantization Artifacts for Achieving Adversarial Outcomes [5.865029600972316]
  Quantization is a technique that transforms the parameter representation of a neural network from floating-point numbers into lower-precision ones. We propose a new training framework to implement adversarial quantization outcomes. We show that a single compromised model defeats multiple quantization schemes.
  arXiv  Detail & Related papers  (2021-10-26T10:09:49Z)
• Mitigating Black-Box Adversarial Attacks via Output Noise Perturbation [4.7591428725916245]
  In black-box adversarial attacks, adversaries query the deep neural network (DNN) In this paper, we study the method of adding white noise to the DNN output to mitigate such attacks.
  arXiv  Detail & Related papers  (2021-09-30T14:24:44Z)
• Detect and Defense Against Adversarial Examples in Deep Learning using Natural Scene Statistics and Adaptive Denoising [12.378017309516965]
  We propose a framework for defending DNN against ad-versarial samples. The detector aims to detect AEs bycharacterizing them through the use of natural scenestatistic. The proposed method outperforms the state-of-the-art defense techniques.
  arXiv  Detail & Related papers  (2021-07-12T23:45:44Z)
• BreakingBED -- Breaking Binary and Efficient Deep Neural Networks by Adversarial Attacks [65.2021953284622]
  We study robustness of CNNs against white-box and black-box adversarial attacks. Results are shown for distilled CNNs, agent-based state-of-the-art pruned models, and binarized neural networks.
  arXiv  Detail & Related papers  (2021-03-14T20:43:19Z)
• Boosting Deep Neural Networks with Geometrical Prior Knowledge: A Survey [77.99182201815763]
  Deep Neural Networks (DNNs) achieve state-of-the-art results in many different problem settings. DNNs are often treated as black box systems, which complicates their evaluation and validation. One promising field, inspired by the success of convolutional neural networks (CNNs) in computer vision tasks, is to incorporate knowledge about symmetric geometrical transformations.
  arXiv  Detail & Related papers  (2020-06-30T14:56:05Z)
• QUANOS- Adversarial Noise Sensitivity Driven Hybrid Quantization of Neural Networks [3.2242513084255036]
  QUANOS is a framework that performs layer-specific hybrid quantization based on Adversarial Noise Sensitivity (ANS) Our experiments on CIFAR10, CIFAR100 datasets show that QUANOS outperforms homogenously quantized 8-bit precision baseline in terms of adversarial robustness.
  arXiv  Detail & Related papers  (2020-04-22T15:56:31Z)
• Adversarial Attacks and Defenses on Graphs: A Review, A Tool and Empirical Studies [73.39668293190019]
  Adversary attacks can be easily fooled by small perturbation on the input. Graph Neural Networks (GNNs) have been demonstrated to inherit this vulnerability. In this survey, we categorize existing attacks and defenses, and review the corresponding state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-03-02T04:32:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.