Zero Memory Overhead Approach for Protecting Vision Transformer Parameters

• URL: http://arxiv.org/abs/2507.03816v1
• Date: Fri, 04 Jul 2025 21:32:24 GMT
• Title: Zero Memory Overhead Approach for Protecting Vision Transformer Parameters
• Authors: Fereshteh Baradaran, Mohsen Raji, Azadeh Baradaran, Arezoo Baradaran, Reihaneh Akbarifard,
• Abstract summary: A fault tolerance technique is introduced to protect ViT parameters against bit-flip faults with zero memory overhead.<n>When faults are detected, affected parameters are masked by zeroing out, as most parameters in ViT models are near zero.<n>This approach enhances reliability across ViT models, improving the robustness of parameters to bit-flips by up to three orders of magnitude.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision Transformers (ViTs) have demonstrated superior performance over Convolutional Neural Networks (CNNs) in various vision-related tasks such as classification, object detection, and segmentation due to their use of self-attention mechanisms. As ViTs become more popular in safety-critical applications like autonomous driving, ensuring their correct functionality becomes essential, especially in the presence of bit-flip faults in their parameters stored in memory. In this paper, a fault tolerance technique is introduced to protect ViT parameters against bit-flip faults with zero memory overhead. Since the least significant bits of parameters are not critical for model accuracy, replacing the LSB with a parity bit provides an error detection mechanism without imposing any overhead on the model. When faults are detected, affected parameters are masked by zeroing out, as most parameters in ViT models are near zero, effectively preventing accuracy degradation. This approach enhances reliability across ViT models, improving the robustness of parameters to bit-flips by up to three orders of magnitude, making it an effective zero-overhead solution for fault tolerance in critical applications.

Related papers

• FT-Transformer: Resilient and Reliable Transformer with End-to-End Fault Tolerant Attention [5.044679241062448]
  Transformer models leverage self-attention mechanisms to capture dependencies, demonstrating exceptional performance in various applications.<n>Existing fault tolerance methods protect each operation separately using decoupled kernels, incurring substantial computational and memory overhead.<n>We propose a novel error-resilient framework for Transformer models, integrating end-to-end fault tolerant attention.
  arXiv  Detail & Related papers  (2025-04-03T02:05:08Z)
• GenBFA: An Evolutionary Optimization Approach to Bit-Flip Attacks on LLMs [3.967858172081495]
  Large Language Models (LLMs) have revolutionized natural language processing (NLP)<n>Increasing adoption in mission-critical applications raises concerns about hardware-based threats, particularly bit-flip attacks (BFAs)
  arXiv  Detail & Related papers  (2024-11-21T00:01:51Z)
• Impactful Bit-Flip Search on Full-precision Models [3.4156622779256995]
  Impactful Bit-Flip Search (IBS) is a novel method for efficiently pinpointing and flipping critical bits in full-precision networks.<n>We propose a Weight-Stealth technique that strategically modifies the model's parameters in a way that maintains the float values within the original distribution.
  arXiv  Detail & Related papers  (2024-11-12T19:24:42Z)
• Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning [91.5113227694443]
  We propose a novel visual. sensuous-aware fine-Tuning (SPT) scheme. SPT allocates trainable parameters to task-specific important positions. Experiments on a wide range of downstream recognition tasks show that our SPT is complementary to the existing PEFT methods.
  arXiv  Detail & Related papers  (2023-03-15T12:34:24Z)
• Soft Error Reliability Analysis of Vision Transformers [14.132398744731635]
  Vision Transformers (ViTs) that leverage self-attention mechanism have shown superior performance on many classical vision tasks. Existing ViTs works mainly optimize performance and accuracy, but ViTs reliability issues induced by soft errors have generally been overlooked. In this work, we study the reliability of ViTs and investigate the vulnerability from different architecture granularities.
  arXiv  Detail & Related papers  (2023-02-21T06:17:40Z)
• Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer [56.87383229709899]
  We develop an information rectification module (IRM) and a distribution guided distillation scheme for fully quantized vision transformers (Q-ViT) Our method achieves a much better performance than the prior arts.
  arXiv  Detail & Related papers  (2022-10-13T04:00:29Z)
• AdaViT: Adaptive Tokens for Efficient Vision Transformer [91.88404546243113]
  We introduce AdaViT, a method that adaptively adjusts the inference cost of vision transformer (ViT) for images of different complexity. AdaViT achieves this by automatically reducing the number of tokens in vision transformers that are processed in the network as inference proceeds.
  arXiv  Detail & Related papers  (2021-12-14T18:56:07Z)
• Global Vision Transformer Pruning with Hessian-Aware Saliency [93.33895899995224]
  This work challenges the common design philosophy of the Vision Transformer (ViT) model with uniform dimension across all the stacked blocks in a model stage. We derive a novel Hessian-based structural pruning criteria comparable across all layers and structures, with latency-aware regularization for direct latency reduction. Performing iterative pruning on the DeiT-Base model leads to a new architecture family called NViT (Novel ViT), with a novel parameter that utilizes parameters more efficiently.
  arXiv  Detail & Related papers  (2021-10-10T18:04:59Z)
• Vision Transformers are Robust Learners [65.91359312429147]
  We study the robustness of the Vision Transformer (ViT) against common corruptions and perturbations, distribution shifts, and natural adversarial examples. We present analyses that provide both quantitative and qualitative indications to explain why ViTs are indeed more robust learners.
  arXiv  Detail & Related papers  (2021-05-17T02:39:22Z)
• TinyTL: Reduce Activations, Not Trainable Parameters for Efficient On-Device Learning [78.80707950262214]
  On-device learning enables edge devices to continually adapt the AI models to new data. Existing work solves this problem by reducing the number of trainable parameters. We present Tiny-Transfer-Learning (TinyTL) for memory-efficient on-device learning.
  arXiv  Detail & Related papers  (2020-07-22T18:39:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.