Filtered-ViT: A Robust Defense Against Multiple Adversarial Patch Attacks

• URL: http://arxiv.org/abs/2511.07755v1
• Date: Wed, 12 Nov 2025 01:15:10 GMT
• Title: Filtered-ViT: A Robust Defense Against Multiple Adversarial Patch Attacks
• Authors: Aja Khanal, Ahmed Faid, Apurva Narayan,
• Abstract summary: Filtered-ViT is a new vision architecture that integrates SMART Median Vector (VMF)<n>On ImageNet with LaVAN multi-patch attacks, Filtered-ViT achieves 79.8% clean accuracy and 46.3% robust accuracy under four simultaneous 1% patches.<n>This establishes Filtered-ViT as the first transformer to demonstrate unified robustness against both adversarial and naturally occurring patch-like disruptions.
• Score: 3.9508022083907393
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning vision systems are increasingly deployed in safety-critical domains such as healthcare, yet they remain vulnerable to small adversarial patches that can trigger misclassifications. Most existing defenses assume a single patch and fail when multiple localized disruptions occur, the type of scenario adversaries and real-world artifacts often exploit. We propose Filtered-ViT, a new vision transformer architecture that integrates SMART Vector Median Filtering (SMART-VMF), a spatially adaptive, multi-scale, robustness-aware mechanism that enables selective suppression of corrupted regions while preserving semantic detail. On ImageNet with LaVAN multi-patch attacks, Filtered-ViT achieves 79.8% clean accuracy and 46.3% robust accuracy under four simultaneous 1\% patches, outperforming existing defenses. Beyond synthetic benchmarks, a real-world case study on radiographic medical imagery shows that Filtered-ViT mitigates natural artifacts such as occlusions and scanner noise without degrading diagnostic content. This establishes Filtered-ViT as the first transformer to demonstrate unified robustness against both adversarial and naturally occurring patch-like disruptions, charting a path toward reliable vision systems in truly high-stakes environments.

Related papers

• Improving Robustness of Vision-Language-Action Models by Restoring Corrupted Visual Inputs [6.2827295422415235]
  Vision-Language-Action (VLA) models have emerged as a dominant paradigm for generalist robotic manipulation.<n>However, reliable real-world deployment is severely hindered by their fragility to visual disturbances.<n>We introduce the Corruption Restoration Transformer (CRT), a vision transformer designed to immunize VLA models against sensor disturbances.
  arXiv  Detail & Related papers  (2026-02-01T11:09:08Z)
• When Robots Obey the Patch: Universal Transferable Patch Attacks on Vision-Language-Action Models [81.7618160628979]
  Vision-Language-Action (VLA) models are vulnerable to adversarial attacks, yet universal and transferable attacks remain underexplored.<n>We introduce UPA-RFAS (Universal Patch Attack via Robust Feature, Attention, and Semantics), a unified framework that learns a single physical patch in a shared feature space.<n> Experiments across diverse VLA models, manipulation suites, and physical executions show that UPA-RFAS consistently transfers across models, tasks, and viewpoints.
  arXiv  Detail & Related papers  (2025-11-26T09:16:32Z)
• Vision Transformers: the threat of realistic adversarial patches [48.03238826812818]
  Vision Transformers (ViTs) have gained significant traction in modern machine learning.<n>ViTs remain vulnerable to evasion attacks, particularly to adversarial patches.<n>This study investigates the transferability of adversarial attack techniques used in CNNs when applied to ViT classification models.
  arXiv  Detail & Related papers  (2025-09-25T12:36:25Z)
• ForensicsSAM: Toward Robust and Unified Image Forgery Detection and Localization Resisting to Adversarial Attack [56.0056378072843]
  We show that highly transferable adversarial images can be crafted solely via the upstream model.<n>We propose ForensicsSAM, a unified IFDL framework with built-in adversarial robustness.
  arXiv  Detail & Related papers  (2025-08-10T16:03:44Z)
• ViTGuard: Attention-aware Detection against Adversarial Examples for Vision Transformer [8.71614629110101]
  We propose ViTGuard as a general detection method for defending Vision Transformer (ViT) models against adversarial attacks. ViTGuard uses a Masked Autoencoder (MAE) model to recover randomly masked patches from the unmasked regions. threshold-based detectors leverage distinctive ViT features, including attention maps and classification (token representations) token representations, to distinguish between normal and adversarial samples.
  arXiv  Detail & Related papers  (2024-09-20T18:11:56Z)
• StealthDiffusion: Towards Evading Diffusion Forensic Detection through Diffusion Model [62.25424831998405]
  StealthDiffusion is a framework that modifies AI-generated images into high-quality, imperceptible adversarial examples. It is effective in both white-box and black-box settings, transforming AI-generated images into high-quality adversarial forgeries.
  arXiv  Detail & Related papers  (2024-08-11T01:22:29Z)
• S-E Pipeline: A Vision Transformer (ViT) based Resilient Classification Pipeline for Medical Imaging Against Adversarial Attacks [4.295229451607423]
  Vision Transformer (ViT) is becoming widely popular in automating accurate disease diagnosis in medical imaging. ViTs remain vulnerable to adversarial attacks that may thwart the diagnosis process by leading it to intentional misclassification of critical disease. We propose a novel image classification pipeline, namely, S-E Pipeline, that performs multiple pre-processing steps.
  arXiv  Detail & Related papers  (2024-07-23T17:20:40Z)
• Towards Robust Vision Transformer via Masked Adaptive Ensemble [23.986968861837813]
  Adversarial training (AT) can help improve the robustness of Vision Transformers (ViT) against adversarial attacks. This paper proposes a novel ViT architecture, including a detector and a classifier bridged by our newly developed adaptive ensemble. Experimental results exhibit that our ViT architecture, on CIFAR-10, achieves the best standard accuracy and adversarial robustness of 90.3% and 49.8%, respectively.
  arXiv  Detail & Related papers  (2024-07-22T05:28:29Z)
• Spatial-Frequency Discriminability for Revealing Adversarial Perturbations [53.279716307171604]
  Vulnerability of deep neural networks to adversarial perturbations has been widely perceived in the computer vision community. Current algorithms typically detect adversarial patterns through discriminative decomposition for natural and adversarial data. We propose a discriminative detector relying on a spatial-frequency Krawtchouk decomposition.
  arXiv  Detail & Related papers  (2023-05-18T10:18:59Z)
• Self-Ensembling Vision Transformer (SEViT) for Robust Medical Image Classification [4.843654097048771]
  Vision Transformers (ViT) are competing to replace Convolutional Neural Networks (CNN) for various computer vision tasks in medical imaging. Recent works have shown that ViTs are also susceptible to such attacks and suffer significant performance degradation under attack. We propose a novel self-ensembling method to enhance the robustness of ViT in the presence of adversarial attacks.
  arXiv  Detail & Related papers  (2022-08-04T19:02:24Z)
• Investigating Robustness of Adversarial Samples Detection for Automatic Speaker Verification [78.51092318750102]
  This work proposes to defend ASV systems against adversarial attacks with a separate detection network. A VGG-like binary classification detector is introduced and demonstrated to be effective on detecting adversarial samples.
  arXiv  Detail & Related papers  (2020-06-11T04:31:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.