EigenShield: Causal Subspace Filtering via Random Matrix Theory for Adversarially Robust Vision-Language Models

• URL: http://arxiv.org/abs/2502.14976v1
• Date: Thu, 20 Feb 2025 19:10:51 GMT
• Title: EigenShield: Causal Subspace Filtering via Random Matrix Theory for Adversarially Robust Vision-Language Models
• Authors: Nastaran Darabi, Devashri Naik, Sina Tayebati, Dinithi Jayasuriya, Ranganath Krishnan, Amit Ranjan Trivedi,
• Abstract summary: Vision-Language Models (VLMs) inherit adversarial vulnerabilities of Large Language Models (LLMs)<n>Existing defenses, including adversarial training, input, and detection, are computationally expensive, architecture-dependent, and fragile against adaptive attacks.<n>We introduce EigenShield, an inference-time defense leveraging Random Matrix Theory to quantify adversarial disruptions in high-dimensional VLM representations.
• Score: 3.958317527488534
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-Language Models (VLMs) inherit adversarial vulnerabilities of Large Language Models (LLMs), which are further exacerbated by their multimodal nature. Existing defenses, including adversarial training, input transformations, and heuristic detection, are computationally expensive, architecture-dependent, and fragile against adaptive attacks. We introduce EigenShield, an inference-time defense leveraging Random Matrix Theory to quantify adversarial disruptions in high-dimensional VLM representations. Unlike prior methods that rely on empirical heuristics, EigenShield employs the spiked covariance model to detect structured spectral deviations. Using a Robustness-based Nonconformity Score (RbNS) and quantile-based thresholding, it separates causal eigenvectors, which encode semantic information, from correlational eigenvectors that are susceptible to adversarial artifacts. By projecting embeddings onto the causal subspace, EigenShield filters adversarial noise without modifying model parameters or requiring adversarial training. This architecture-independent, attack-agnostic approach significantly reduces the attack success rate, establishing spectral analysis as a principled alternative to conventional defenses. Our results demonstrate that EigenShield consistently outperforms all existing defenses, including adversarial training, UNIGUARD, and CIDER.

Related papers

• MOREL: Enhancing Adversarial Robustness through Multi-Objective Representation Learning [1.534667887016089]
  deep neural networks (DNNs) are vulnerable to slight adversarial perturbations.<n>We show that strong feature representation learning during training can significantly enhance the original model's robustness.<n>We propose MOREL, a multi-objective feature representation learning approach, encouraging classification models to produce similar features for inputs within the same class, despite perturbations.
  arXiv  Detail & Related papers  (2024-10-02T16:05:03Z)
• Efficient Adversarial Training in LLMs with Continuous Attacks [99.5882845458567]
  Large language models (LLMs) are vulnerable to adversarial attacks that can bypass their safety guardrails. We propose a fast adversarial training algorithm (C-AdvUL) composed of two losses. C-AdvIPO is an adversarial variant of IPO that does not require utility data for adversarially robust alignment.
  arXiv  Detail & Related papers  (2024-05-24T14:20:09Z)
• Meta Invariance Defense Towards Generalizable Robustness to Unknown Adversarial Attacks [62.036798488144306]
  Current defense mainly focuses on the known attacks, but the adversarial robustness to the unknown attacks is seriously overlooked. We propose an attack-agnostic defense method named Meta Invariance Defense (MID) We show that MID simultaneously achieves robustness to the imperceptible adversarial perturbations in high-level image classification and attack-suppression in low-level robust image regeneration.
  arXiv  Detail & Related papers  (2024-04-04T10:10:38Z)
• PuriDefense: Randomized Local Implicit Adversarial Purification for Defending Black-box Query-based Attacks [15.842917276255141]
  Black-box query-based attacks threaten Machine Learning as a Service (ML) systems. We propose an efficient defense mechanism, PuriDefense, that employs random patch-wise purifications with an ensemble of lightweight purification models at a low level of inference cost. Our theoretical analysis suggests that this approach slows down the convergence of query-based attacks by incorporating randomness into purifications.
  arXiv  Detail & Related papers  (2024-01-19T09:54:23Z)
• Defense Against Model Extraction Attacks on Recommender Systems [53.127820987326295]
  We introduce Gradient-based Ranking Optimization (GRO) to defend against model extraction attacks on recommender systems. GRO aims to minimize the loss of the protected target model while maximizing the loss of the attacker's surrogate model. Results show GRO's superior effectiveness in defending against model extraction attacks.
  arXiv  Detail & Related papers  (2023-10-25T03:30:42Z)
• Carefully Blending Adversarial Training, Purification, and Aggregation Improves Adversarial Robustness [1.2289361708127877]
  CARSO is able to defend itself against adaptive end-to-end white-box attacks devised for defences.<n>Our method improves by a significant margin the state-of-the-art for Cifar-10, Cifar-100, and TinyImageNet-200.
  arXiv  Detail & Related papers  (2023-05-25T09:04:31Z)
• Improving Adversarial Robustness to Sensitivity and Invariance Attacks with Deep Metric Learning [80.21709045433096]
  A standard method in adversarial robustness assumes a framework to defend against samples crafted by minimally perturbing a sample. We use metric learning to frame adversarial regularization as an optimal transport problem. Our preliminary results indicate that regularizing over invariant perturbations in our framework improves both invariant and sensitivity defense.
  arXiv  Detail & Related papers  (2022-11-04T13:54:02Z)
• Resisting Adversarial Attacks in Deep Neural Networks using Diverse Decision Boundaries [12.312877365123267]
  Deep learning systems are vulnerable to crafted adversarial examples, which may be imperceptible to the human eye, but can lead the model to misclassify. We develop a new ensemble-based solution that constructs defender models with diverse decision boundaries with respect to the original model. We present extensive experimentations using standard image classification datasets, namely MNIST, CIFAR-10 and CIFAR-100 against state-of-the-art adversarial attacks.
  arXiv  Detail & Related papers  (2022-08-18T08:19:26Z)
• Bilateral Dependency Optimization: Defending Against Model-inversion Attacks [61.78426165008083]
  We propose a bilateral dependency optimization (BiDO) strategy to defend against model-inversion attacks. BiDO achieves the state-of-the-art defense performance for a variety of datasets, classifiers, and MI attacks.
  arXiv  Detail & Related papers  (2022-06-11T10:07:03Z)
• Clustering Effect of (Linearized) Adversarial Robust Models [60.25668525218051]
  We propose a novel understanding of adversarial robustness and apply it on more tasks including domain adaption and robustness boosting. Experimental evaluations demonstrate the rationality and superiority of our proposed clustering strategy.
  arXiv  Detail & Related papers  (2021-11-25T05:51:03Z)
• Generating Out of Distribution Adversarial Attack using Latent Space Poisoning [5.1314136039587925]
  We propose a novel mechanism of generating adversarial examples where the actual image is not corrupted. latent space representation is utilized to tamper with the inherent structure of the image. As opposed to gradient-based attacks, the latent space poisoning exploits the inclination of classifiers to model the independent and identical distribution of the training dataset.
  arXiv  Detail & Related papers  (2020-12-09T13:05:44Z)
• TREND: Transferability based Robust ENsemble Design [6.663641564969944]
  We study the effect of network architecture, input, weight and activation quantization on transferability of adversarial samples. We show that transferability is significantly hampered by input quantization between source and target. We propose a new state-of-the-art ensemble attack to combat this.
  arXiv  Detail & Related papers  (2020-08-04T13:38:14Z)
• Adversarial Distributional Training for Robust Deep Learning [53.300984501078126]
  Adversarial training (AT) is among the most effective techniques to improve model robustness by augmenting training data with adversarial examples. Most existing AT methods adopt a specific attack to craft adversarial examples, leading to the unreliable robustness against other unseen attacks. In this paper, we introduce adversarial distributional training (ADT), a novel framework for learning robust models.
  arXiv  Detail & Related papers  (2020-02-14T12:36:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.