SARIF: Segment Anything for Robust Image Forensics

• URL: http://arxiv.org/abs/2606.21108v1
• Date: Fri, 19 Jun 2026 05:21:48 GMT
• Title: SARIF: Segment Anything for Robust Image Forensics
• Abstract summary: We propose SARIF (Segment Anything for Robust Image Forensics), a framework that leverages the Segment Anything Model (SAM)<n> SARIF introduces a feedback-guided mask decoder and a dual-encoder design that extracts forgery-specific information to capture forensic traces.<n>Experiments on standard forgery-localization benchmarks show that SARIF achieves strong average cross-dataset performance and robustness to common image corruptions.
• Score: 2.9917768336120196
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image forgery localization remains challenging due to diverse manipulation techniques and distribution shifts. Existing forgery localization models achieve high accuracy on benchmarks but often struggle with cross-domain generalization and robustness. In this paper, we propose SARIF (Segment Anything for Robust Image Forensics), a framework that leverages the Segment Anything Model (SAM), which has a promptable architecture and strong generalization ability. SARIF introduces a feedback-guided mask decoder and a dual-encoder design that extracts forgery-specific information to capture forensic traces while exploiting the SAM architecture. To localize manipulated regions, we design a block-wise prompting mechanism that derives forgery-specific cues from residual features between an adapted encoder and its frozen counterpart. These features are fused with the previous mask prompt to drive a feedback-based mask refinement process, enabling automatic forgery segmentation without manual input. Extensive experiments on standard forgery-localization benchmarks show that SARIF achieves strong average cross-dataset performance and robustness to common image corruptions.

Related papers

• From Detection to Localization: A Unified Forensics Framework for Fully Synthetic and Tampered Images [7.9180538960840385]
  This work extends an existing detector by introducing a unified multiclass framework (real vs. fully generated vs. tampered)<n>In addition to classifying image authenticity, the framework incorporates a segmentation branch to enable pixel-level localization of tampered regions.<n>The proposed approach outperforms selected recent benchmarks, offering an efficient solution with improved classification accuracy and higher IoU scores for the localization task.
  arXiv  Detail & Related papers  (2026-09-02T14:16:44Z)
• FUSED: Forensic-Semantic Mixture-of-Experts for AI Inpainting Detection and Localization [14.056602265859645]
  Diffusion-based inpainting models modify only a localized part of an image.<n>Many AI-image detectors rely on global artifacts and do not localize.<n>We present FUSED, a unified framework for the joint detection and localization of AI-generated inpainting.
  arXiv  Detail & Related papers  (2026-08-28T13:05:43Z)
• ForensicsTok: Forensics-Guided Tokenized Modeling for Image Tampering Localization [31.83661814822312]
  We propose ForensicsTok, which reformulates image manipulation localization as an autoregressive sequence generation task.<n>ForensicsTok directly generates spatially grounded token sequences, enabling precise mask prediction without intermediary supervision.
  arXiv  Detail & Related papers  (2026-06-23T13:06:18Z)
• PR-MaGIC: Prompt Refinement Via Mask Decoder Gradient Flow For In-Context Segmentation [12.227797182463611]
  We introduce PR-MaGIC (Prompt Refinement via Mask Decoder for In-Context), a training-free test-time framework that refines prompts via gradient flow from SAM's mask decoder.<n> PR-MaGIC consistently improves segmentation quality across various benchmarks, effectively mitigating inadequate prompts without requiring additional training or architectural modifications.
  arXiv  Detail & Related papers  (2026-04-13T22:40:04Z)
• Excite, Attend and Segment (EASe): Domain-Agnostic Fine-Grained Mask Discovery with Feature Calibration and Self-Supervised Upsampling [0.0]
  Excite, Attend and Segment (EASe) is an unsupervised domain-agnostic semantic segmentation framework.<n>Our evaluation demonstrates superior performance of EASe over previous state-of-the-arts.
  arXiv  Detail & Related papers  (2026-03-31T22:03:15Z)
• When Detectors Forget Forensics: Blocking Semantic Shortcuts for Generalizable AI-Generated Image Detection [48.75021863792367]
  Vision Foundation Models (VFMs, emphe.g., CLIP) often struggle to generalize to images created using unseen generation pipelines.<n>We propose textbfGeometric Semantic Decoupling (GSD), a parameter-free module that explicitly removes semantic components from learned representations.<n>GSD estimates semantic directions from batch-wise statistics and projects them out via a geometric constraint, forcing the artifact detector to rely on semantic-invariant forensic evidence.
  arXiv  Detail & Related papers  (2026-03-10T06:16:35Z)
• SIGMAE: A Spectral-Index-Guided Foundation Model for Multispectral Remote Sensing [43.39478017496301]
  Masked Autoencoder (MAE)-based pretraining stands out for its strong capability to learn general feature representations via reconstructing masked image regions.<n>Applying MAE to multispectral remote sensing images remains challenging due to complex backgrounds, indistinct targets, and the lack of semantic guidance during masking.<n> SIGMAE incorporates domain-specific spectral indices as prior knowledge to guide dynamic token masking toward informative regions.
  arXiv  Detail & Related papers  (2026-03-08T04:55:41Z)
• ForensicsSAM: Toward Robust and Unified Image Forgery Detection and Localization Resisting to Adversarial Attack [56.0056378072843]
  We show that highly transferable adversarial images can be crafted solely via the upstream model.<n>We propose ForensicsSAM, a unified IFDL framework with built-in adversarial robustness.
  arXiv  Detail & Related papers  (2025-08-10T16:03:44Z)
• Context-Aware Weakly Supervised Image Manipulation Localization with SAM Refinement [52.15627062770557]
  Malicious image manipulation poses societal risks, increasing the importance of effective image manipulation detection methods.<n>Recent approaches in image manipulation detection have largely been driven by fully supervised approaches.<n>We present a novel weakly supervised framework based on a dual-branch Transformer-CNN architecture.
  arXiv  Detail & Related papers  (2025-03-26T07:35:09Z)
• Towards Fine-grained Interactive Segmentation in Images and Videos [21.22536962888316]
  We present an SAM2Refiner framework built upon the SAM2 backbone.<n>This architecture allows SAM2 to generate fine-grained segmentation masks for both images and videos.<n>In addition, a mask refinement module is devised by employing a multi-scale cascaded structure to fuse mask features with hierarchical representations from the encoder.
  arXiv  Detail & Related papers  (2025-02-12T06:38:18Z)
• A Large-scale Interpretable Multi-modality Benchmark for Facial Image Forgery Localization [22.725542948364357]
  We argue that the basic binary forgery mask is inadequate for explaining model predictions.<n>In this study, we generate salient region-focused interpretation for the forgery images.<n>We develop ForgeryTalker, an architecture designed for concurrent forgery localization and interpretation.
  arXiv  Detail & Related papers  (2024-12-27T15:23:39Z)
• Guided Interpretable Facial Expression Recognition via Spatial Action Unit Cues [55.97779732051921]
  A new learning strategy is proposed to explicitly incorporate au cues into classifier training. We show that our strategy can improve layer-wise interpretability without degrading classification performance.
  arXiv  Detail & Related papers  (2024-02-01T02:13:49Z)
• Progressive Feature Self-reinforcement for Weakly Supervised Semantic Segmentation [55.69128107473125]
  We propose a single-stage approach for Weakly Supervised Semantic (WSSS) with image-level labels. We adaptively partition the image content into deterministic regions (e.g., confident foreground and background) and uncertain regions (e.g., object boundaries and misclassified categories) for separate processing. Building upon this, we introduce a complementary self-enhancement method that constrains the semantic consistency between these confident regions and an augmented image with the same class labels.
  arXiv  Detail & Related papers  (2023-12-14T13:21:52Z)
• Improving Adversarial Robustness of Masked Autoencoders via Test-time Frequency-domain Prompting [133.55037976429088]
  We investigate the adversarial robustness of vision transformers equipped with BERT pretraining (e.g., BEiT, MAE) A surprising observation is that MAE has significantly worse adversarial robustness than other BERT pretraining methods. We propose a simple yet effective way to boost the adversarial robustness of MAE.
  arXiv  Detail & Related papers  (2023-08-20T16:27:17Z)
• Boosting Few-shot Semantic Segmentation with Transformers [81.43459055197435]
  TRansformer-based Few-shot Semantic segmentation method (TRFS) Our model consists of two modules: Global Enhancement Module (GEM) and Local Enhancement Module (LEM)
  arXiv  Detail & Related papers  (2021-08-04T20:09:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.