Quantitative Analysis of Deeply Quantized Tiny Neural Networks Robust to Adversarial Attacks

• URL: http://arxiv.org/abs/2503.08973v1
• Date: Wed, 12 Mar 2025 00:34:25 GMT
• Title: Quantitative Analysis of Deeply Quantized Tiny Neural Networks Robust to Adversarial Attacks
• Authors: Idris Zakariyya, Ferheen Ayaz, Mounia Kharbouche-Harrari, Jeremy Singer, Sye Loong Keoh, Danilo Pau, José Cano,
• Abstract summary: A drawback of Deep Neural Networks (DNNs) is their susceptibility to adversarial attacks.<n>This paper presents the outcomes of a compact DNN model that exhibits resilience against both black-box and white-box adversarial attacks.<n>It has achieved this resilience through training with the QKeras quantization-aware training framework.
• Score: 1.6975640673527588
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Reducing the memory footprint of Machine Learning (ML) models, especially Deep Neural Networks (DNNs), is imperative to facilitate their deployment on resource-constrained edge devices. However, a notable drawback of DNN models lies in their susceptibility to adversarial attacks, wherein minor input perturbations can deceive them. A primary challenge revolves around the development of accurate, resilient, and compact DNN models suitable for deployment on resource-constrained edge devices. This paper presents the outcomes of a compact DNN model that exhibits resilience against both black-box and white-box adversarial attacks. This work has achieved this resilience through training with the QKeras quantization-aware training framework. The study explores the potential of QKeras and an adversarial robustness technique, Jacobian Regularization (JR), to co-optimize the DNN architecture through per-layer JR methodology. As a result, this paper has devised a DNN model employing this co-optimization strategy based on Stochastic Ternary Quantization (STQ). Its performance was compared against existing DNN models in the face of various white-box and black-box attacks. The experimental findings revealed that, the proposed DNN model had small footprint and on average, it exhibited better performance than Quanos and DS-CNN MLCommons/TinyML (MLC/T) benchmarks when challenged with white-box and black-box attacks, respectively, on the CIFAR-10 image and Google Speech Commands audio datasets.

Related papers

• Make Me a BNN: A Simple Strategy for Estimating Bayesian Uncertainty from Pre-trained Models [40.38541033389344]
  Deep Neural Networks (DNNs) are powerful tools for various computer vision tasks, yet they often struggle with reliable uncertainty quantification. We introduce the Adaptable Bayesian Neural Network (ABNN), a simple and scalable strategy to seamlessly transform DNNs into BNNs. We conduct extensive experiments across multiple datasets for image classification and semantic segmentation tasks, and our results demonstrate that ABNN achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-12-23T16:39:24Z)
• Securing Graph Neural Networks in MLaaS: A Comprehensive Realization of Query-based Integrity Verification [68.86863899919358]
  We introduce a groundbreaking approach to protect GNN models in Machine Learning from model-centric attacks. Our approach includes a comprehensive verification schema for GNN's integrity, taking into account both transductive and inductive GNNs. We propose a query-based verification technique, fortified with innovative node fingerprint generation algorithms.
  arXiv  Detail & Related papers  (2023-12-13T03:17:05Z)
• Improving the Robustness of Quantized Deep Neural Networks to White-Box Attacks using Stochastic Quantization and Information-Theoretic Ensemble Training [1.6098666134798774]
  Most real-world applications that employ deep neural networks (DNNs) quantize them to low precision to reduce the compute needs. We present a method to improve the robustness of quantized DNNs to white-box adversarial attacks.
  arXiv  Detail & Related papers  (2023-11-30T17:15:58Z)
• Improving Robustness Against Adversarial Attacks with Deeply Quantized Neural Networks [0.5849513679510833]
  A disadvantage of Deep Neural Networks (DNNs) is their vulnerability to adversarial attacks, as they can be fooled by adding slight perturbations to the inputs. This paper reports the results of devising a tiny DNN model, robust to adversarial black and white box attacks, trained with an automatic quantizationaware training framework.
  arXiv  Detail & Related papers  (2023-04-25T13:56:35Z)
• Robust and Lossless Fingerprinting of Deep Neural Networks via Pooled Membership Inference [17.881686153284267]
  Deep neural networks (DNNs) have already achieved great success in a lot of application areas and brought profound changes to our society. How to protect the intellectual property (IP) of DNNs against infringement is one of the most important yet very challenging topics. This paper proposes a novel technique called emphpooled membership inference (PMI) so as to protect the IP of the DNN models.
  arXiv  Detail & Related papers  (2022-09-09T04:06:29Z)
• Recurrent Bilinear Optimization for Binary Neural Networks [58.972212365275595]
  BNNs neglect the intrinsic bilinear relationship of real-valued weights and scale factors. Our work is the first attempt to optimize BNNs from the bilinear perspective. We obtain robust RBONNs, which show impressive performance over state-of-the-art BNNs on various models and datasets.
  arXiv  Detail & Related papers  (2022-09-04T06:45:33Z)
• Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation [70.75043144299168]
  Spiking Neural Network (SNN) is a promising energy-efficient AI model when implemented on neuromorphic hardware. It is a challenge to efficiently train SNNs due to their non-differentiability. We propose the Differentiation on Spike Representation (DSR) method, which could achieve high performance.
  arXiv  Detail & Related papers  (2022-05-01T12:44:49Z)
• Robustness of Bayesian Neural Networks to White-Box Adversarial Attacks [55.531896312724555]
  Bayesian Networks (BNNs) are robust and adept at handling adversarial attacks by incorporating randomness. We create our BNN model, called BNN-DenseNet, by fusing Bayesian inference (i.e., variational Bayes) to the DenseNet architecture. An adversarially-trained BNN outperforms its non-Bayesian, adversarially-trained counterpart in most experiments.
  arXiv  Detail & Related papers  (2021-11-16T16:14:44Z)
• KNN-BERT: Fine-Tuning Pre-Trained Models with KNN Classifier [61.063988689601416]
  Pre-trained models are widely used in fine-tuning downstream tasks with linear classifiers optimized by the cross-entropy loss. These problems can be improved by learning representations that focus on similarities in the same class and contradictions when making predictions. We introduce the KNearest Neighbors in pre-trained model fine-tuning tasks in this paper.
  arXiv  Detail & Related papers  (2021-10-06T06:17:05Z)
• BreakingBED -- Breaking Binary and Efficient Deep Neural Networks by Adversarial Attacks [65.2021953284622]
  We study robustness of CNNs against white-box and black-box adversarial attacks. Results are shown for distilled CNNs, agent-based state-of-the-art pruned models, and binarized neural networks.
  arXiv  Detail & Related papers  (2021-03-14T20:43:19Z)
• An Integrated Approach to Produce Robust Models with High Efficiency [9.476463361600828]
  Quantization and structure simplification are promising ways to adapt Deep Neural Networks (DNNs) to mobile devices. In this work, we try to obtain both features by applying a convergent relaxation quantization algorithm, Binary-Relax (BR), to a robust adversarial-trained model, ResNets Ensemble. We design a trade-off loss function that helps DNNs preserve their natural accuracy and improve the channel sparsity.
  arXiv  Detail & Related papers  (2020-08-31T00:44:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.