FUSED: Forensic-Semantic Mixture-of-Experts for AI Inpainting Detection and Localization

• URL: http://arxiv.org/abs/2608.28302v1
• Date: Fri, 28 Aug 2026 13:05:43 GMT
• Title: FUSED: Forensic-Semantic Mixture-of-Experts for AI Inpainting Detection and Localization
• Abstract summary: Diffusion-based inpainting models modify only a localized part of an image.<n>Many AI-image detectors rely on global artifacts and do not localize.<n>We present FUSED, a unified framework for the joint detection and localization of AI-generated inpainting.
• Score: 14.056602265859645
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Diffusion-based inpainting models modify only a localized part of an image, while many AI-image detectors rely on global artifacts and do not localize. These artifacts vary across generators, limiting detector transfer under distribution shifts. Recent work shows that restoring the authentic pixels outside the inpainted region removes these cues and can degrade pretrained detectors. To address this, we present FUSED, a unified framework for the joint detection and localization of AI-generated inpainting. FUSED combines low-level forensic cues with high-level semantic features using a sparsely-gated Mixture-of-Experts architecture, enabling the model to adaptively prioritize the most relevant signal for each token. For each input, FUSED predicts both an image-level manipulation score and a pixel-level mask of the inpainted area. On the OpenSDID cross-generator benchmark, FUSED achieves the best average detection and localization, with the largest gains on unseen generators. The same model transfers directly to the held-out AutoSplice and CocoGlide benchmarks, more than doubling localization performance. Evaluating each held-out benchmark with and without the global generator artifact further shows that all evaluated methods, ours included, partly read the artifact as evidence of manipulation, and FUSED remains the strongest under both conditions. Code and pretrained models are available at https://github.com/AntonNuzhdin/FUSED.

Related papers

• From Detection to Localization: A Unified Forensics Framework for Fully Synthetic and Tampered Images [7.9180538960840385]
  This work extends an existing detector by introducing a unified multiclass framework (real vs. fully generated vs. tampered)<n>In addition to classifying image authenticity, the framework incorporates a segmentation branch to enable pixel-level localization of tampered regions.<n>The proposed approach outperforms selected recent benchmarks, offering an efficient solution with improved classification accuracy and higher IoU scores for the localization task.
  arXiv  Detail & Related papers  (2026-09-02T14:16:44Z)
• GAP-SAM: A Global Artifact Prior for Generalizable AI-Generated Image Manipulation Localization [15.115841637929895]
  We construct COCO-ControlNet with source-image Canny edges and depth maps to align semantics and geometry, improving OOD performance across multiple localizers.<n>We also identify emphboundary adhesion, where fine-tuned segmentation models snap predictions to semantic object contours rather than true manipulation boundaries.<n>GAP-SAM encodes an image and its frozen VAE reconstruction into a global artifact token and injects it into SAM3's feature pyramid via zero-gated FiLM before pixel decoding.
  arXiv  Detail & Related papers  (2026-08-21T09:49:58Z)
• LoRC: Detecting AI-Generated Images via Low-Rank Collapse in Semantic Residuals [50.157719466558696]
  We show that modern generators exhibit low-rank collapse in the semantic-residual subspace while preserving the dominant semantic direction.<n>We propose textbfLoRC, a framework that decouples semantic dominance to capture the collapsed residual geometry.<n>Our method improves accuracy by an average of 7.0% across multiple benchmarks and achieves 97.0% accuracy on 39 unseen generators.
  arXiv  Detail & Related papers  (2026-08-21T08:58:47Z)
• When Detectors Forget Forensics: Blocking Semantic Shortcuts for Generalizable AI-Generated Image Detection [48.75021863792367]
  Vision Foundation Models (VFMs, emphe.g., CLIP) often struggle to generalize to images created using unseen generation pipelines.<n>We propose textbfGeometric Semantic Decoupling (GSD), a parameter-free module that explicitly removes semantic components from learned representations.<n>GSD estimates semantic directions from batch-wise statistics and projects them out via a geometric constraint, forcing the artifact detector to rely on semantic-invariant forensic evidence.
  arXiv  Detail & Related papers  (2026-03-10T06:16:35Z)
• Unveiling Perceptual Artifacts: A Fine-Grained Benchmark for Interpretable AI-Generated Image Detection [95.08316274158165]
  X-AIGD provides pixel-level, categorized annotations of perceptual artifacts, spanning low-level distortions, high-level semantics, and cognitive-level counterfactuals.<n>Existing AIGI detectors demonstrate negligible reliance on perceptual artifacts, even at the most basic distortion level.<n>Explicitly aligning model attention with artifact regions can increase the interpretability and generalization of detectors.
  arXiv  Detail & Related papers  (2026-01-27T10:09:17Z)
• CINEMAE: Leveraging Frozen Masked Autoencoders for Cross-Generator AI Image Detection [25.84217122259626]
  CINEMAE adapts the core principles of text detection methods to the visual domain.<n>We trained exclusively on Stable Diffusion v1.4, our method achieves over 95% accuracy on all eight unseen generators in the GenImage benchmark.<n>This demonstrates that context-conditional reconstruction uncertainty provides a robust, transferable signal for AIGC detection.
  arXiv  Detail & Related papers  (2025-11-09T11:05:45Z)
• Zooming In on Fakes: A Novel Dataset for Localized AI-Generated Image Detection with Forgery Amplification Approach [69.01456182499486]
  textbfBR-Gen is a large-scale dataset of 150,000 locally forged images with diverse scene-aware annotations.<n>textbfNFA-ViT is a Noise-guided Forgery Amplification Vision Transformer that enhances the detection of localized forgeries.
  arXiv  Detail & Related papers  (2025-04-16T09:57:23Z)
• Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection [86.97062579515833]
  We introduce the concept of Neighboring Pixel Relationships(NPR) as a means to capture and characterize the generalized structural artifacts stemming from up-sampling operations. A comprehensive analysis is conducted on an open-world dataset, comprising samples generated by tft28 distinct generative models. This analysis culminates in the establishment of a novel state-of-the-art performance, showcasing a remarkable tft11.6% improvement over existing methods.
  arXiv  Detail & Related papers  (2023-12-16T14:27:06Z)
• Weakly-supervised deepfake localization in diffusion-generated images [4.548755617115687]
  We propose a weakly-supervised localization problem based on the Xception network as the backbone architecture. We show that the best performing detection method (based on local scores) is less sensitive to the looser supervision than to the mismatch in terms of dataset or generator.
  arXiv  Detail & Related papers  (2023-11-08T10:27:36Z)
• Pixel-Inconsistency Modeling for Image Manipulation Localization [59.968362815126326]
  Digital image forensics plays a crucial role in image authentication and manipulation localization. This paper presents a generalized and robust manipulation localization model through the analysis of pixel inconsistency artifacts. Experiments show that our method successfully extracts inherent pixel-inconsistency forgery fingerprints.
  arXiv  Detail & Related papers  (2023-09-30T02:54:51Z)
• TruFor: Leveraging all-round clues for trustworthy image forgery detection and localization [17.270110456445806]
  TruFor is a forensic framework that can be applied to a large variety of image manipulation methods. We rely on the extraction of both high-level and low-level traces through a transformer-based fusion architecture. Our method is able to reliably detect and localize both cheapfakes and deepfakes manipulations outperforming state-of-the-art works.
  arXiv  Detail & Related papers  (2022-12-21T11:49:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.