ForensicsSAM: Toward Robust and Unified Image Forgery Detection and Localization Resisting to Adversarial Attack

• URL: http://arxiv.org/abs/2508.07402v2
• Date: Sun, 17 Aug 2025 08:03:11 GMT
• Title: ForensicsSAM: Toward Robust and Unified Image Forgery Detection and Localization Resisting to Adversarial Attack
• Authors: Rongxuan Peng, Shunquan Tan, Chenqi Kong, Anwei Luo, Alex C. Kot, Jiwu Huang,
• Abstract summary: We show that highly transferable adversarial images can be crafted solely via the upstream model.<n>We propose ForensicsSAM, a unified IFDL framework with built-in adversarial robustness.
• Score: 56.0056378072843
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Parameter-efficient fine-tuning (PEFT) has emerged as a popular strategy for adapting large vision foundation models, such as the Segment Anything Model (SAM) and LLaVA, to downstream tasks like image forgery detection and localization (IFDL). However, existing PEFT-based approaches overlook their vulnerability to adversarial attacks. In this paper, we show that highly transferable adversarial images can be crafted solely via the upstream model, without accessing the downstream model or training data, significantly degrading the IFDL performance. To address this, we propose ForensicsSAM, a unified IFDL framework with built-in adversarial robustness. Our design is guided by three key ideas: (1) To compensate for the lack of forgery-relevant knowledge in the frozen image encoder, we inject forgery experts into each transformer block to enhance its ability to capture forgery artifacts. These forgery experts are always activated and shared across any input images. (2) To detect adversarial images, we design an light-weight adversary detector that learns to capture structured, task-specific artifact in RGB domain, enabling reliable discrimination across various attack methods. (3) To resist adversarial attacks, we inject adversary experts into the global attention layers and MLP modules to progressively correct feature shifts induced by adversarial noise. These adversary experts are adaptively activated by the adversary detector, thereby avoiding unnecessary interference with clean images. Extensive experiments across multiple benchmarks demonstrate that ForensicsSAM achieves superior resistance to various adversarial attack methods, while also delivering state-of-the-art performance in image-level forgery detection and pixel-level forgery localization. The resource is available at https://github.com/siriusPRX/ForensicsSAM.

Related papers

• Universal Anti-forensics Attack against Image Forgery Detection via Multi-modal Guidance [22.94094331220455]
  ForgeryEraser is a framework designed to execute universal anti-forensics attack without access to the target AIGC detectors.<n>We show that ForgeryEraser causes substantial performance degradation to advanced AIGC detectors on global synthesis and local editing benchmarks.
  arXiv  Detail & Related papers  (2026-02-06T09:32:10Z)
• Active Adversarial Noise Suppression for Image Forgery Localization [56.98050814363447]
  We introduce an Adversarial Noise Suppression Module (ANSM) that generate a defensive perturbation to suppress the attack effect of adversarial noise.<n>To our best knowledge, this is the first report of adversarial defense in image forgery localization tasks.
  arXiv  Detail & Related papers  (2025-06-15T14:53:27Z)
• Adversarially Robust AI-Generated Image Detection for Free: An Information Theoretic Perspective [22.514709685678813]
  We show that adversarial training (AT) suffers from performance collapse in AIGI detection.<n>Motivated by this difference, we propose Training-free Robust Detection via Information-theoretic Measures (TRIM)<n>TRIM builds on standard detectors and quantifies feature shifts using prediction entropy and KL divergence.
  arXiv  Detail & Related papers  (2025-05-28T17:20:49Z)
• StealthDiffusion: Towards Evading Diffusion Forensic Detection through Diffusion Model [62.25424831998405]
  StealthDiffusion is a framework that modifies AI-generated images into high-quality, imperceptible adversarial examples. It is effective in both white-box and black-box settings, transforming AI-generated images into high-quality adversarial forgeries.
  arXiv  Detail & Related papers  (2024-08-11T01:22:29Z)
• Vulnerabilities in AI-generated Image Detection: The Challenge of Adversarial Attacks [39.524974831780874]
  We show that adversarial attack is truly a real threat to AIGI detectors, because FPBA can deliver successful black-box attacks.<n>We name our method as Frequency-based Post-train Bayesian Attack, or FPBA.
  arXiv  Detail & Related papers  (2024-07-30T14:07:17Z)
• Evading Detection Actively: Toward Anti-Forensics against Forgery Localization [40.124726174594024]
  Anti-forensics seeks to eliminate or conceal traces of tampering artifacts. Traditional adversarial attack methods cannot be directly used against forgery localization. We propose SEAR (Self-supErvised Anti-foRensics), a novel self-supervised and adversarial training algorithm.
  arXiv  Detail & Related papers  (2023-10-16T03:44:10Z)
• PAIF: Perception-Aware Infrared-Visible Image Fusion for Attack-Tolerant Semantic Segmentation [50.556961575275345]
  We propose a perception-aware fusion framework to promote segmentation robustness in adversarial scenes. We show that our scheme substantially enhances the robustness, with gains of 15.3% mIOU, compared with advanced competitors.
  arXiv  Detail & Related papers  (2023-08-08T01:55:44Z)
• Adversarially-Aware Robust Object Detector [85.10894272034135]
  We propose a Robust Detector (RobustDet) based on adversarially-aware convolution to disentangle gradients for model learning on clean and adversarial images. Our model effectively disentangles gradients and significantly enhances the detection robustness with maintaining the detection ability on clean images.
  arXiv  Detail & Related papers  (2022-07-13T13:59:59Z)
• Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Driving [87.3492357041748]
  In this paper, we showcase practical susceptibilities of multi-sensor detection by placing an adversarial object on top of a host vehicle. Our experiments demonstrate that successful attacks are primarily caused by easily corrupted image features. Towards more robust multi-modal perception systems, we show that adversarial training with feature denoising can boost robustness to such attacks significantly.
  arXiv  Detail & Related papers  (2021-01-17T21:15:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.