LoRA as a Flexible Framework for Securing Large Vision Systems

• URL: http://arxiv.org/abs/2506.00661v2
• Date: Thu, 03 Jul 2025 18:01:09 GMT
• Title: LoRA as a Flexible Framework for Securing Large Vision Systems
• Authors: Zander W. Blasingame, Richard E. Neddo, Chen Liu,
• Abstract summary: Adversarial attacks have emerged as a critical threat to autonomous driving systems.<n>We propose to take insights for parameter efficient fine-tuning and use low-rank adaptation (LoRA) to train a lightweight security patch.<n>We demonstrate that our framework can patch a pre-trained model to improve classification accuracy by up to 78.01% in the presence of adversarial examples.
• Score: 1.9035583634286277
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adversarial attacks have emerged as a critical threat to autonomous driving systems. These attacks exploit the underlying neural network, allowing small -- nearly invisible -- perturbations to completely alter the behavior of such systems in potentially malicious ways. E.g., causing a traffic sign classification network to misclassify a stop sign as a speed limit sign. Prior working in hardening such systems to adversarial attacks have looked at robust training of the system or adding additional pre-processing steps to the input pipeline. Such solutions either have a hard time generalizing, require knowledge of the adversarial attacks during training, or are computationally undesirable. Instead, we propose to take insights for parameter efficient fine-tuning and use low-rank adaptation (LoRA) to train a lightweight security patch -- enabling us to dynamically patch a large preexisting vision system as new vulnerabilities are discovered. We demonstrate that our framework can patch a pre-trained model to improve classification accuracy by up to 78.01% in the presence of adversarial examples.

Related papers

• Robust Anti-Backdoor Instruction Tuning in LVLMs [53.766434746801366]
  We introduce a lightweight, certified-agnostic defense framework for large visual language models (LVLMs)<n>Our framework finetunes only adapter modules and text embedding layers under instruction tuning.<n>Experiments against seven attacks on Flickr30k and MSCOCO demonstrate that ours reduces their attack success rate to nearly zero.
  arXiv  Detail & Related papers  (2025-06-04T01:23:35Z)
• AttentionGuard: Transformer-based Misbehavior Detection for Secure Vehicular Platoons [0.0]
  Vehicle platooning is vulnerable to sophisticated falsification attacks by authenticated insiders.<n>We present AttentionGuard, a transformer-based framework for misbehavior detection.<n>We show that AttentionGuard achieves up to 0.95 F1-score in attack detection, with robust performance maintained during complex maneuvers.
  arXiv  Detail & Related papers  (2025-05-15T13:24:09Z)
• Improving the Shortest Plank: Vulnerability-Aware Adversarial Training for Robust Recommender System [60.719158008403376]
  Vulnerability-aware Adversarial Training (VAT) is designed to defend against poisoning attacks in recommender systems. VAT employs a novel vulnerability-aware function to estimate users' vulnerability based on the degree to which the system fits them.
  arXiv  Detail & Related papers  (2024-09-26T02:24:03Z)
• Downstream Transfer Attack: Adversarial Attacks on Downstream Models with Pre-trained Vision Transformers [95.22517830759193]
  This paper studies the transferability of such an adversarial vulnerability from a pre-trained ViT model to downstream tasks. We show that DTA achieves an average attack success rate (ASR) exceeding 90%, surpassing existing methods by a huge margin.
  arXiv  Detail & Related papers  (2024-08-03T08:07:03Z)
• Transform-Dependent Adversarial Attacks [15.374381635334897]
  We introduce transform-dependent adversarial attacks on deep networks.<n>Our perturbations exhibit metamorphic properties, enabling diverse adversarial effects as a function of transformation parameters.<n>We show that transform-dependent perturbations achieve high targeted attack success rates, outperforming state-of-the-art transfer attacks by 17-31% in blackbox scenarios.
  arXiv  Detail & Related papers  (2024-06-12T17:31:36Z)
• FaultGuard: A Generative Approach to Resilient Fault Prediction in Smart Electrical Grids [53.2306792009435]
  FaultGuard is the first framework for fault type and zone classification resilient to adversarial attacks. We propose a low-complexity fault prediction model and an online adversarial training technique to enhance robustness. Our model outclasses the state-of-the-art for resilient fault prediction benchmarking, with an accuracy of up to 0.958.
  arXiv  Detail & Related papers  (2024-03-26T08:51:23Z)
• AutoAugment Input Transformation for Highly Transferable Targeted Attacks [9.970326131028159]
  We propose a novel targeted adversarial attack called AutoAugment Input Transformation (AAIT) AAIT searches for the optimal transformation policy from a transformation space comprising various operations. It crafts adversarial examples using the found optimal transformation policy to boost the adversarial transferability in targeted attacks.
  arXiv  Detail & Related papers  (2023-12-21T12:49:36Z)
• Attention Deficit is Ordered! Fooling Deformable Vision Transformers with Collaborative Adversarial Patches [3.4673556247932225]
  Deformable vision transformers significantly reduce the complexity of attention modeling. Recent work has demonstrated adversarial attacks against conventional vision transformers. We develop new collaborative attacks where a source patch manipulates attention to point to a target patch, which contains the adversarial noise to fool the model.
  arXiv  Detail & Related papers  (2023-11-21T17:55:46Z)
• When Authentication Is Not Enough: On the Security of Behavioral-Based Driver Authentication Systems [53.2306792009435]
  We develop two lightweight driver authentication systems based on Random Forest and Recurrent Neural Network architectures. We are the first to propose attacks against these systems by developing two novel evasion attacks, SMARTCAN and GANCAN. Through our contributions, we aid practitioners in safely adopting these systems, help reduce car thefts, and enhance driver security.
  arXiv  Detail & Related papers  (2023-06-09T14:33:26Z)
• Enhancing the Self-Universality for Transferable Targeted Attacks [88.6081640779354]
  Our new attack method is proposed based on the observation that highly universal adversarial perturbations tend to be more transferable for targeted attacks. Instead of optimizing the perturbations on different images, optimizing on different regions to achieve self-universality can get rid of using extra data. With the feature similarity loss, our method makes the features from adversarial perturbations to be more dominant than that of benign images.
  arXiv  Detail & Related papers  (2022-09-08T11:21:26Z)
• Unrestricted Adversarial Attacks on ImageNet Competition [70.8952435964555]
  Unrestricted adversarial attack is popular and practical direction but has not been studied thoroughly. We organize this competition with the purpose of exploring more effective unrestricted adversarial attack algorithm.
  arXiv  Detail & Related papers  (2021-10-17T04:27:15Z)
• Towards Transferable Adversarial Attacks on Vision Transformers [110.55845478440807]
  Vision transformers (ViTs) have demonstrated impressive performance on a series of computer vision tasks, yet they still suffer from adversarial examples. We introduce a dual attack framework, which contains a Pay No Attention (PNA) attack and a PatchOut attack, to improve the transferability of adversarial samples across different ViTs.
  arXiv  Detail & Related papers  (2021-09-09T11:28:25Z)
• The Feasibility and Inevitability of Stealth Attacks [63.14766152741211]
  We study new adversarial perturbations that enable an attacker to gain control over decisions in generic Artificial Intelligence systems. In contrast to adversarial data modification, the attack mechanism we consider here involves alterations to the AI system itself.
  arXiv  Detail & Related papers  (2021-06-26T10:50:07Z)
• Combating Adversaries with Anti-Adversaries [118.70141983415445]
  In particular, our layer generates an input perturbation in the opposite direction of the adversarial one. We verify the effectiveness of our approach by combining our layer with both nominally and robustly trained models. Our anti-adversary layer significantly enhances model robustness while coming at no cost on clean accuracy.
  arXiv  Detail & Related papers  (2021-03-26T09:36:59Z)
• Model-Agnostic Defense for Lane Detection against Adversarial Attack [0.0]
  Recent work on adversarial road patches have successfully induced perception of lane lines with arbitrary form. We propose a modular lane verification system that can catch such threats before the autonomous driving system is misled. Our experiments show that implementing the system with a simple convolutional neural network (CNN) can defend against a wide gamut of attacks on lane detection models.
  arXiv  Detail & Related papers  (2021-03-01T00:05:50Z)
• End-to-end Uncertainty-based Mitigation of Adversarial Attacks to Automated Lane Centering [12.11406399284803]
  We propose an end-to-end approach that addresses the impact of adversarial attacks throughout perception, planning, and control modules. Our approach can effectively mitigate the impact of adversarial attacks and can achieve 55% to 90% improvement over the original OpenPilot.
  arXiv  Detail & Related papers  (2021-02-27T22:36:32Z)
• Online Alternate Generator against Adversarial Attacks [144.45529828523408]
  Deep learning models are notoriously sensitive to adversarial examples which are synthesized by adding quasi-perceptible noises on real images. We propose a portable defense method, online alternate generator, which does not need to access or modify the parameters of the target networks. The proposed method works by online synthesizing another image from scratch for an input image, instead of removing or destroying adversarial noises.
  arXiv  Detail & Related papers  (2020-09-17T07:11:16Z)
• A Self-supervised Approach for Adversarial Robustness [105.88250594033053]
  Adversarial examples can cause catastrophic mistakes in Deep Neural Network (DNNs) based vision systems. This paper proposes a self-supervised adversarial training mechanism in the input space. It provides significant robustness against the textbfunseen adversarial attacks.
  arXiv  Detail & Related papers  (2020-06-08T20:42:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.