GAP-SAM: A Global Artifact Prior for Generalizable AI-Generated Image Manipulation Localization

• URL: http://arxiv.org/abs/2608.20929v1
• Date: Fri, 21 Aug 2026 09:49:58 GMT
• Title: GAP-SAM: A Global Artifact Prior for Generalizable AI-Generated Image Manipulation Localization
• Abstract summary: We construct COCO-ControlNet with source-image Canny edges and depth maps to align semantics and geometry, improving OOD performance across multiple localizers.<n>We also identify emphboundary adhesion, where fine-tuned segmentation models snap predictions to semantic object contours rather than true manipulation boundaries.<n>GAP-SAM encodes an image and its frozen VAE reconstruction into a global artifact token and injects it into SAM3's feature pyramid via zero-gated FiLM before pixel decoding.
• Score: 15.115841637929895
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: AI-generated image manipulation localization identifies edited pixels, but its OOD performance lags behind image-level detection partly because pixel supervision entangles forensic evidence with dataset-specific mask geometry and semantic boundaries. Extending image-level distribution alignment to localization, we construct COCO-ControlNet with source-image Canny edges and depth maps to align semantics and geometry, improving OOD performance across multiple localizers. Yet tighter Mask-VAE Reconstruction Alignment (Mask-VAE) underperforms COCO-ControlNet, showing that VAE reconstruction artifacts transfer poorly to local diffusion-inpainting artifacts. We also identify \emph{boundary adhesion}, where fine-tuned segmentation models snap predictions to semantic object contours rather than true manipulation boundaries. These findings motivate GAP-SAM, which encodes an image and its frozen VAE reconstruction into a global artifact token and injects it into SAM3's feature pyramid via zero-gated FiLM before pixel decoding. Without prescribing a spatial region, this token modulates dense decoding to preserve localization while suppressing semantic-boundary shortcuts. Across six datasets, GAP-SAM averages 79.8 Pixel-F1, outperforming the strongest prior method by 12.6 points. It also performs best at every tested severity of JPEG compression, Gaussian blur, and resizing.

Related papers

• FUSED: Forensic-Semantic Mixture-of-Experts for AI Inpainting Detection and Localization [14.056602265859645]
  Diffusion-based inpainting models modify only a localized part of an image.<n>Many AI-image detectors rely on global artifacts and do not localize.<n>We present FUSED, a unified framework for the joint detection and localization of AI-generated inpainting.
  arXiv  Detail & Related papers  (2026-08-28T13:05:43Z)
• SIGMA: Semantic-Difference Instruction-Grounding Mask Annotator for Text-Driven Image Manipulation Localization [64.6372217552272]
  Public editing datasets contain millions of structurally identical (original, edited) pairs to IML training samples, lacking only pixel-level masks.<n>We propose SIGMA, which performs semantic-feature differencing in a vision foundation backbone.<n>It produces a 1.1M IML training set that improves six diverse detectors by +18.34% F1 across five datasets.
  arXiv  Detail & Related papers  (2026-05-27T03:55:13Z)
• From Semantics to Pixels: Coarse-to-Fine Masked Autoencoders for Hierarchical Visual Understanding [31.516613298007005]
  C2FMAE is a coarse-to-fine masked autoencoder that explicitly learns hierarchical visual representations across three data granularities.<n>We show that C2FMAE achieves significant performance gains on image classification, object detection, and semantic segmentation.
  arXiv  Detail & Related papers  (2026-03-10T17:51:12Z)
• Gaussian Masked Autoencoders [74.2341070024126]
  This paper explores Masked Autoencoders (MAE) with Gaussian Splatting.<n>Our approach, named Gaussian Masked Autoencoder, or GMAE, aims to learn semantic abstractions and spatial understanding jointly.
  arXiv  Detail & Related papers  (2025-01-06T18:59:57Z)
• Pixel-Inconsistency Modeling for Image Manipulation Localization [59.968362815126326]
  Digital image forensics plays a crucial role in image authentication and manipulation localization. This paper presents a generalized and robust manipulation localization model through the analysis of pixel inconsistency artifacts. Experiments show that our method successfully extracts inherent pixel-inconsistency forgery fingerprints.
  arXiv  Detail & Related papers  (2023-09-30T02:54:51Z)
• In-Domain GAN Inversion for Faithful Reconstruction and Editability [132.68255553099834]
  We propose in-domain GAN inversion, which consists of a domain-guided domain-regularized and a encoder to regularize the inverted code in the native latent space of the pre-trained GAN model. We make comprehensive analyses on the effects of the encoder structure, the starting inversion point, as well as the inversion parameter space, and observe the trade-off between the reconstruction quality and the editing property.
  arXiv  Detail & Related papers  (2023-09-25T08:42:06Z)
• iEdit: Localised Text-guided Image Editing with Weak Supervision [53.082196061014734]
  We propose a novel learning method for text-guided image editing. It generates images conditioned on a source image and a textual edit prompt. It shows favourable results against its counterparts in terms of image fidelity, CLIP alignment score and qualitatively for editing both generated and real images.
  arXiv  Detail & Related papers  (2023-05-10T07:39:14Z)
• CoupAlign: Coupling Word-Pixel with Sentence-Mask Alignments for Referring Image Segmentation [104.5033800500497]
  Referring image segmentation aims at localizing all pixels of the visual objects described by a natural language sentence. Previous works learn to straightforwardly align the sentence embedding and pixel-level embedding for highlighting the referred objects. We propose CoupAlign, a simple yet effective multi-level visual-semantic alignment method.
  arXiv  Detail & Related papers  (2022-12-04T08:53:42Z)
• An Empirical Method to Quantify the Peripheral Performance Degradation in Deep Networks [18.808132632482103]
  convolutional neural network (CNN) kernels compound with each convolutional layer. Deeper and deeper networks combined with stride-based down-sampling means that the propagation of this region can end up covering a non-negligable portion of the image. Our dataset is constructed by inserting objects into high resolution backgrounds, thereby allowing us to crop sub-images which place target objects at specific locations relative to the image border. By probing the behaviour of Mask R-CNN across a selection of target locations, we see clear patterns of performance degredation near the image boundary, and in particular in the image corners.
  arXiv  Detail & Related papers  (2020-12-04T18:00:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.