SIFT-Graph: Benchmarking Multimodal Defense Against Image Adversarial Attacks With Robust Feature Graph

• URL: http://arxiv.org/abs/2511.08810v1
• Date: Thu, 13 Nov 2025 01:09:36 GMT
• Title: SIFT-Graph: Benchmarking Multimodal Defense Against Image Adversarial Attacks With Robust Feature Graph
• Abstract summary: Adversarial attacks expose a fundamental vulnerability in modern deep vision models.<n>We introduce SIFT-Graph, a multimodal defense framework that enhances the robustness of traditional vision models.<n>Preliminary results demonstrate that our method effectively improves the visual model robustness against gradient-based white box adversarial attacks.
• Score: 0.6999740786886536
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adversarial attacks expose a fundamental vulnerability in modern deep vision models by exploiting their dependence on dense, pixel-level representations that are highly sensitive to imperceptible perturbations. Traditional defense strategies typically operate within this fragile pixel domain, lacking mechanisms to incorporate inherently robust visual features. In this work, we introduce SIFT-Graph, a multimodal defense framework that enhances the robustness of traditional vision models by aggregating structurally meaningful features extracted from raw images using both handcrafted and learned modalities. Specifically, we integrate Scale-Invariant Feature Transform keypoints with a Graph Attention Network to capture scale and rotation invariant local structures that are resilient to perturbations. These robust feature embeddings are then fused with traditional vision model, such as Vision Transformer and Convolutional Neural Network, to form a unified, structure-aware and perturbation defensive model. Preliminary results demonstrate that our method effectively improves the visual model robustness against gradient-based white box adversarial attacks, while incurring only a marginal drop in clean accuracy.

Related papers

• Exposing Vulnerabilities in Visible-Infrared VLMs: A Unified Geometric Adversarial Framework with Cross-Task Transferability [11.461885964494199]
  Vision-language models (VLMs) have achieved strong performance across diverse multimodal tasks, but their adversarial robustness in visible-infrared (VIS-IR) scenarios remains underexplored.<n>We propose CFGPatch, a curved-edge fractal geometric adversarial patch framework for attacking VIS-IR VLMs.
  arXiv  Detail & Related papers  (2026-05-21T10:15:58Z)
• Physically-Induced Atmospheric Adversarial Perturbations: Enhancing Transferability and Robustness in Remote Sensing Image Classification [54.952203312050564]
  Adversarial attacks pose a severe threat to the reliability of deep learning models in remote sensing (RS) image classification.<n>We propose FogFool, a physically plausible adversarial framework that generates fog-based perturbations.<n>Experiments on two benchmark RS datasets demonstrate that FogFool achieves superior performance.
  arXiv  Detail & Related papers  (2026-04-16T05:47:57Z)
• Fixed-point graph convolutional networks against adversarial attacks [2.4469484645516832]
  Adversarial attacks present a significant risk to the integrity and performance of graph neural networks.<n>We present a novel model, called fixed-point iterative graph convolutional network iteration (Fix-GCN), which achieves robustness against adversarial perturbations.
  arXiv  Detail & Related papers  (2025-10-29T20:17:37Z)
• Foundations and Models in Modern Computer Vision: Key Building Blocks in Landmark Architectures [34.542592986038265]
  This report analyzes the evolution of key design patterns in computer vision by examining six influential papers.<n>We review ResNet, which introduced residual connections to overcome the vanishing gradient problem.<n>We examine the Vision Transformer (ViT), which established a new paradigm by applying the Transformer architecture to sequences of image patches.
  arXiv  Detail & Related papers  (2025-07-31T09:08:11Z)
• Vision Foundation Models as Effective Visual Tokenizers for Autoregressive Image Generation [52.261584726401686]
  We present a novel direction to build an image tokenizer directly on top of a frozen vision foundation model.<n>Based on these designs, our proposed image tokenizer, VFMTok, achieves substantial improvements in image reconstruction and generation quality.
  arXiv  Detail & Related papers  (2025-07-11T09:32:45Z)
• Graph Defense Diffusion Model [26.41730982598055]
  Graph Neural Networks (GNNs) are highly vulnerable to adversarial attacks, which can greatly degrade their performance.<n>Existing graph purification methods attempt to address this issue by filtering attacked graphs.<n>We propose a more versatile approach for defending against adversarial attacks on graphs.
  arXiv  Detail & Related papers  (2025-01-20T16:18:40Z)
• Locally-Focused Face Representation for Sketch-to-Image Generation Using Noise-Induced Refinement [1.7409266903306055]
  This paper presents a novel deep-learning framework that significantly enhances the transformation of rudimentary face sketches into high-fidelity colour images.<n>Our approach effectively captures and enhances critical facial features through a block attention mechanism within an encoder-decoder architecture.<n>The model sets a new state-of-the-art in sketch-to-image generation, can generalize across sketch types, and offers a robust solution for applications such as criminal identification in law enforcement.
  arXiv  Detail & Related papers  (2024-11-28T09:12:56Z)
• Robust Network Learning via Inverse Scale Variational Sparsification [55.64935887249435]
  We introduce an inverse scale variational sparsification framework within a time-continuous inverse scale space formulation. Unlike frequency-based methods, our approach not only removes noise by smoothing small-scale features. We show the efficacy of our approach through enhanced robustness against various noise types.
  arXiv  Detail & Related papers  (2024-09-27T03:17:35Z)
• Harnessing The Power of Attention For Patch-Based Biomedical Image Classification [0.0]
  We present a novel architecture based on self-attention mechanisms as an alternative to conventional CNNs. We introduce the Lancoz5 technique, which adapts variable image sizes to higher resolutions. Our methods address critical challenges faced by attention-based vision models, including inductive bias, weight sharing, receptive field limitations, and efficient data handling.
  arXiv  Detail & Related papers  (2024-04-01T06:22:28Z)
• Manipulating Feature Visualizations with Gradient Slingshots [53.94925202421929]
  Feature Visualization (FV) is a widely used technique for interpreting the concepts learned by Deep Neural Networks (DNNs)<n>We introduce a novel method, Gradient Slingshots, that enables manipulation of FV without modifying the model architecture or significantly degrading its performance.
  arXiv  Detail & Related papers  (2024-01-11T18:57:17Z)
• Distance Weighted Trans Network for Image Completion [52.318730994423106]
  We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components. CNNs are used to augment the local texture information of coarse priors. DWT blocks are used to recover certain coarse textures and coherent visual structures.
  arXiv  Detail & Related papers  (2023-10-11T12:46:11Z)
• Encoding Robustness to Image Style via Adversarial Feature Perturbations [72.81911076841408]
  We adapt adversarial training by directly perturbing feature statistics, rather than image pixels, to produce robust models. Our proposed method, Adversarial Batch Normalization (AdvBN), is a single network layer that generates worst-case feature perturbations during training.
  arXiv  Detail & Related papers  (2020-09-18T17:52:34Z)
• Limited-angle tomographic reconstruction of dense layered objects by dynamical machine learning [68.9515120904028]
  Limited-angle tomography of strongly scattering quasi-transparent objects is a challenging, highly ill-posed problem. Regularizing priors are necessary to reduce artifacts by improving the condition of such problems. We devised a recurrent neural network (RNN) architecture with a novel split-convolutional gated recurrent unit (SC-GRU) as the building block.
  arXiv  Detail & Related papers  (2020-07-21T11:48:22Z)
• Single Image Deraining via Scale-space Invariant Attention Neural Network [58.5284246878277]
  We tackle the notion of scale that deals with visual changes in appearance of rain steaks with respect to the camera. We propose to represent the multi-scale correlation in convolutional feature domain, which is more compact and robust than that in pixel domain. In this way, we summarize the most activated presence of feature maps as the salient features.
  arXiv  Detail & Related papers  (2020-06-09T04:59:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.