MIGA: Mutual Information-Guided Attack on Denoising Models for Semantic Manipulation

• URL: http://arxiv.org/abs/2503.06966v2
• Date: Tue, 11 Mar 2025 06:01:25 GMT
• Title: MIGA: Mutual Information-Guided Attack on Denoising Models for Semantic Manipulation
• Authors: Guanghao Li, Mingzhi Chen, Hao Yu, Shuting Dong, Wenhao Jiang, Ming Tang, Chun Yuan,
• Abstract summary: We propose Mutual Information-Guided Attack (MIGA) to directly attack deep denoising models.<n>MIGA strategically disrupts denoising models' ability to preserve semantic content via adversarial perturbations.<n>Our findings suggest that denoising models are not always robust and can introduce security risks in real-world applications.
• Score: 39.12448251986432
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Deep learning-based denoising models have been widely employed in vision tasks, functioning as filters to eliminate noise while retaining crucial semantic information. Additionally, they play a vital role in defending against adversarial perturbations that threaten downstream tasks. However, these models can be intrinsically susceptible to adversarial attacks due to their dependence on specific noise assumptions. Existing attacks on denoising models mainly aim at deteriorating visual clarity while neglecting semantic manipulation, rendering them either easily detectable or limited in effectiveness. In this paper, we propose Mutual Information-Guided Attack (MIGA), the first method designed to directly attack deep denoising models by strategically disrupting their ability to preserve semantic content via adversarial perturbations. By minimizing the mutual information between the original and denoised images, a measure of semantic similarity. MIGA forces the denoiser to produce perceptually clean yet semantically altered outputs. While these images appear visually plausible, they encode systematically distorted semantics, revealing a fundamental vulnerability in denoising models. These distortions persist in denoised outputs and can be quantitatively assessed through downstream task performance. We propose new evaluation metrics and systematically assess MIGA on four denoising models across five datasets, demonstrating its consistent effectiveness in disrupting semantic fidelity. Our findings suggest that denoising models are not always robust and can introduce security risks in real-world applications.

Related papers

• A Knowledge-guided Adversarial Defense for Resisting Malicious Visual Manipulation [93.28532038721816]
  Malicious applications of visual manipulation have raised serious threats to the security and reputation of users in many fields. We propose a knowledge-guided adversarial defense (KGAD) to actively force malicious manipulation models to output semantically confusing samples.
  arXiv  Detail & Related papers  (2025-04-11T10:18:13Z)
• Anti-Reference: Universal and Immediate Defense Against Reference-Based Generation [24.381813317728195]
  Anti-Reference is a novel method that protects images from the threats posed by reference-based generation techniques.<n>We propose a unified loss function that enables joint attacks on fine-tuning-based customization methods.<n>Our method shows certain transfer attack capabilities, effectively challenging both gray-box models and some commercial APIs.
  arXiv  Detail & Related papers  (2024-12-08T16:04:45Z)
• Transferable Adversarial Attacks on SAM and Its Downstream Models [87.23908485521439]
  This paper explores the feasibility of adversarial attacking various downstream models fine-tuned from the segment anything model (SAM) To enhance the effectiveness of the adversarial attack towards models fine-tuned on unknown datasets, we propose a universal meta-initialization (UMI) algorithm.
  arXiv  Detail & Related papers  (2024-10-26T15:04:04Z)
• Rethinking and Defending Protective Perturbation in Personalized Diffusion Models [21.30373461975769]
  We study the fine-tuning process of personalized diffusion models (PDMs) through the lens of shortcut learning. PDMs are susceptible to minor adversarial perturbations, leading to significant degradation when fine-tuned on corrupted datasets. We propose a systematic defense framework that includes data purification and contrastive decoupling learning.
  arXiv  Detail & Related papers  (2024-06-27T07:14:14Z)
• Low-Trace Adaptation of Zero-shot Self-supervised Blind Image Denoising [23.758547513866766]
  We propose a trace-constraint loss function and low-trace adaptation Noise2Noise (LoTA-N2N) model to bridge the gap between self-supervised and supervised learning. Our method achieves state-of-the-art performance within the realm of zero-shot self-supervised image denoising approaches.
  arXiv  Detail & Related papers  (2024-03-19T02:47:33Z)
• Stable Unlearnable Example: Enhancing the Robustness of Unlearnable Examples via Stable Error-Minimizing Noise [31.586389548657205]
  Unlearnable example is proposed to significantly degrade the generalization performance of models by adding a kind of imperceptible noise to the data. We introduce stable error-minimizing noise (SEM), which trains the defensive noise against random perturbation instead of the time-consuming adversarial perturbation. SEM achieves a new state-of-the-art performance on CIFAR-10, CIFAR-100, and ImageNet Subset.
  arXiv  Detail & Related papers  (2023-11-22T01:43:57Z)
• Evaluating Similitude and Robustness of Deep Image Denoising Models via Adversarial Attack [60.40356882897116]
  Deep neural networks (DNNs) have shown superior performance compared to traditional image denoising algorithms. In this paper, we propose an adversarial attack method named denoising-PGD which can successfully attack all the current deep denoising models.
  arXiv  Detail & Related papers  (2023-06-28T09:30:59Z)
• Beyond Pretrained Features: Noisy Image Modeling Provides Adversarial Defense [52.66971714830943]
  Masked image modeling (MIM) has made it a prevailing framework for self-supervised visual representation learning. In this paper, we investigate how this powerful self-supervised learning paradigm can provide adversarial robustness to downstream classifiers. We propose an adversarial defense method, referred to as De3, by exploiting the pretrained decoder for denoising.
  arXiv  Detail & Related papers  (2023-02-02T12:37:24Z)
• Towards Adversarially Robust Deep Image Denoising [199.2458715635285]
  This work systematically investigates the adversarial robustness of deep image denoisers (DIDs) We propose a novel adversarial attack, namely Observation-based Zero-mean Attack (sc ObsAtk) to craft adversarial zero-mean perturbations on given noisy images. To robustify DIDs, we propose hybrid adversarial training (sc HAT) that jointly trains DIDs with adversarial and non-adversarial noisy data.
  arXiv  Detail & Related papers  (2022-01-12T10:23:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.