AICAttack: Adversarial Image Captioning Attack with Attention-Based Optimization

• URL: http://arxiv.org/abs/2402.11940v3
• Date: Thu, 5 Sep 2024 05:06:57 GMT
• Title: AICAttack: Adversarial Image Captioning Attack with Attention-Based Optimization
• Authors: Jiyao Li, Mingze Ni, Yifei Dong, Tianqing Zhu, Wei Liu,
• Abstract summary: This paper presents a novel adversarial attack strategy, AICAttack, designed to attack image captioning models through subtle perturbations on images. operating within a black-box attack scenario, our algorithm requires no access to the target model's architecture, parameters, or gradient information. We demonstrate AICAttack's effectiveness through extensive experiments on benchmark datasets against multiple victim models.
• Score: 13.045125782574306
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in deep learning research have shown remarkable achievements across many tasks in computer vision (CV) and natural language processing (NLP). At the intersection of CV and NLP is the problem of image captioning, where the related models' robustness against adversarial attacks has not been well studied. This paper presents a novel adversarial attack strategy, AICAttack (Attention-based Image Captioning Attack), designed to attack image captioning models through subtle perturbations on images. Operating within a black-box attack scenario, our algorithm requires no access to the target model's architecture, parameters, or gradient information. We introduce an attention-based candidate selection mechanism that identifies the optimal pixels to attack, followed by a customised differential evolution method to optimise the perturbations of pixels' RGB values. We demonstrate AICAttack's effectiveness through extensive experiments on benchmark datasets against multiple victim models. The experimental results demonstrate that our method outperforms current leading-edge techniques by achieving consistently higher attack success rates.

Related papers

• MirrorCheck: Efficient Adversarial Defense for Vision-Language Models [55.73581212134293]
  We propose a novel, yet elegantly simple approach for detecting adversarial samples in Vision-Language Models. Our method leverages Text-to-Image (T2I) models to generate images based on captions produced by target VLMs. Empirical evaluations conducted on different datasets validate the efficacy of our approach.
  arXiv  Detail & Related papers  (2024-06-13T15:55:04Z)
• SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentation [56.622250514119294]
  In contrast to white-box adversarial attacks, transfer attacks are more reflective of real-world scenarios. We propose a self-augment-based transfer attack method, termed SA-Attack.
  arXiv  Detail & Related papers  (2023-12-08T09:08:50Z)
• Adv-Attribute: Inconspicuous and Transferable Adversarial Attack on Face Recognition [111.1952945740271]
  Adversarial Attributes (Adv-Attribute) is designed to generate inconspicuous and transferable attacks on face recognition. Experiments on the FFHQ and CelebA-HQ datasets show that the proposed Adv-Attribute method achieves the state-of-the-art attacking success rates.
  arXiv  Detail & Related papers  (2022-10-13T09:56:36Z)
• Meta Adversarial Perturbations [66.43754467275967]
  We show the existence of a meta adversarial perturbation (MAP) MAP causes natural images to be misclassified with high probability after being updated through only a one-step gradient ascent update. We show that these perturbations are not only image-agnostic, but also model-agnostic, as a single perturbation generalizes well across unseen data points and different neural network architectures.
  arXiv  Detail & Related papers  (2021-11-19T16:01:45Z)
• Geometrically Adaptive Dictionary Attack on Face Recognition [23.712389625037442]
  We propose a strategy for query-efficient black-box attacks on face recognition. Our core idea is to create an adversarial perturbation in the UV texture map and project it onto the face in the image. We show overwhelming performance improvement in the experiments on the LFW and CPLFW datasets.
  arXiv  Detail & Related papers  (2021-11-08T10:26:28Z)
• Bridge the Gap Between CV and NLP! A Gradient-based Textual Adversarial Attack Framework [17.17479625646699]
  We propose a unified framework to craft textual adversarial samples. In this paper, we instantiate our framework with an attack algorithm named Textual Projected Gradient Descent (T-PGD)
  arXiv  Detail & Related papers  (2021-10-28T17:31:51Z)
• Deep Image Destruction: A Comprehensive Study on Vulnerability of Deep Image-to-Image Models against Adversarial Attacks [104.8737334237993]
  We present comprehensive investigations into the vulnerability of deep image-to-image models to adversarial attacks. For five popular image-to-image tasks, 16 deep models are analyzed from various standpoints. We show that unlike in image classification tasks, the performance degradation on image-to-image tasks can largely differ depending on various factors.
  arXiv  Detail & Related papers  (2021-04-30T14:20:33Z)
• AdvHaze: Adversarial Haze Attack [19.744435173861785]
  We introduce a novel adversarial attack method based on haze, which is a common phenomenon in real-world scenery. Our method can synthesize potentially adversarial haze into an image based on the atmospheric scattering model with high realisticity. We demonstrate that the proposed method achieves a high success rate, and holds better transferability across different classification models than the baselines.
  arXiv  Detail & Related papers  (2021-04-28T09:52:25Z)
• PICA: A Pixel Correlation-based Attentional Black-box Adversarial Attack [37.15301296824337]
  We propose a pixel correlation-based attentional black-box adversarial attack, termed as PICA. PICA is more efficient to generate high-resolution adversarial examples compared with the existing black-box attacks.
  arXiv  Detail & Related papers  (2021-01-19T09:53:52Z)
• Reinforcement Learning-based Black-Box Evasion Attacks to Link Prediction in Dynamic Graphs [87.5882042724041]
  Link prediction in dynamic graphs (LPDG) is an important research problem that has diverse applications. We study the vulnerability of LPDG methods and propose the first practical black-box evasion attack.
  arXiv  Detail & Related papers  (2020-09-01T01:04:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.