Bridging the Micro--Macro Gap: Frequency-Aware Semantic Alignment for Image Manipulation Localization

• URL: http://arxiv.org/abs/2604.12341v1
• Date: Tue, 14 Apr 2026 06:24:12 GMT
• Title: Bridging the Micro--Macro Gap: Frequency-Aware Semantic Alignment for Image Manipulation Localization
• Abstract summary: We propose FASA, a unified framework for localizing both traditional and diffusion-generated manipulations.<n>Experiments demonstrate state-of-the-art localization performance, strong cross-dataset generalization, and robust performance under common image degradations.
• Score: 8.02155804287918
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As generative image editing advances, image manipulation localization (IML) must handle both traditional manipulations with conspicuous forensic artifacts and diffusion-generated edits that appear locally realistic. Existing methods typically rely on either low-level forensic cues or high-level semantics alone, leading to a fundamental micro--macro gap. To bridge this gap, we propose FASA, a unified framework for localizing both traditional and diffusion-generated manipulations. Specifically, we extract manipulation-sensitive frequency cues through an adaptive dual-band DCT module and learn manipulation-aware semantic priors via patch-level contrastive alignment on frozen CLIP representations. We then inject these priors into a hierarchical frequency pathway through a semantic-frequency side adapter for multi-scale feature interaction, and employ a prototype-guided, frequency-gated mask decoder to integrate semantic consistency with boundary-aware localization for tampered region prediction. Extensive experiments on OpenSDI and multiple traditional manipulation benchmarks demonstrate state-of-the-art localization performance, strong cross-generator and cross-dataset generalization, and robust performance under common image degradations.

Related papers

• FUSED: Forensic-Semantic Mixture-of-Experts for AI Inpainting Detection and Localization [14.056602265859645]
  Diffusion-based inpainting models modify only a localized part of an image.<n>Many AI-image detectors rely on global artifacts and do not localize.<n>We present FUSED, a unified framework for the joint detection and localization of AI-generated inpainting.
  arXiv  Detail & Related papers  (2026-08-28T13:05:43Z)
• SC-Diff: Semantically Calibrated Diffusion for Visible-to-Infrared Image Translation [52.62632888060867]
  Visible-to-infrared image translation provides a practical way to expand infrared training data using abundant visible images.<n>Existing diffusion-based methods typically use semantic priors only as external conditions.<n>We propose SC-Diff, a semantically latent diffusion framework that uses semantic priors for both conditional guidance and internal self-attention calibration.
  arXiv  Detail & Related papers  (2026-08-09T07:57:40Z)
• FOCA: Frequency-Oriented Cross-Domain Forgery Detection, Localization and Explanation via Multi-Modal Large Language Model [11.08248067961235]
  FOCA is a large language model-based framework that integrates discriminative features from both the RGB spatial and frequency domains.<n>FSE-Set is a large-scale dataset with diverse authentic and tampered images, pixel-level masks, and dual-domain annotations.
  arXiv  Detail & Related papers  (2026-02-21T15:53:44Z)
• Detecting AI-Generated Forgeries via Iterative Manifold Deviation Amplification [11.465428962772245]
  We propose IFA-Net, which shifts from learning "what is fake" to modeling "what is real"<n>Experiments on four diffusion-based inpainting benchmarks show that IFA-Net achieves an average improvement of 6.5% in IoU and 8.1% in F1-score over the second-best method.
  arXiv  Detail & Related papers  (2026-02-21T14:14:17Z)
• Contrastive Learning with Diffusion Features for Weakly Supervised Medical Image Segmentation [12.530950480385554]
  Conditional diffusion model (CDM) has been used as an alternative for generating segmentation masks.<n>We introduce Contrastive Learning with Diffusion Features (CLDF) to train a pixel decoder to map the diffusion features from a frozen CDM to a low-dimensional embedding space for segmentation.
  arXiv  Detail & Related papers  (2025-06-30T01:43:50Z)
• Freqformer: Image-Demoiréing Transformer via Efficient Frequency Decomposition [83.40450475728792]
  We present Freqformer, a Transformer-based framework specifically designed for image demoir'eing through targeted frequency separation.<n>Our method performs an effective frequency decomposition that explicitly splits moir'e patterns into high-frequency spatially-localized textures and low-frequency scale-robust color distortions.<n>Experiments on various demoir'eing benchmarks demonstrate that Freqformer achieves state-of-the-art performance with a compact model size.
  arXiv  Detail & Related papers  (2025-05-25T12:23:10Z)
• Context-Aware Weakly Supervised Image Manipulation Localization with SAM Refinement [52.15627062770557]
  Malicious image manipulation poses societal risks, increasing the importance of effective image manipulation detection methods.<n>Recent approaches in image manipulation detection have largely been driven by fully supervised approaches.<n>We present a novel weakly supervised framework based on a dual-branch Transformer-CNN architecture.
  arXiv  Detail & Related papers  (2025-03-26T07:35:09Z)
• Mixture-of-Noises Enhanced Forgery-Aware Predictor for Multi-Face Manipulation Detection and Localization [52.87635234206178]
  This paper proposes a new framework, namely MoNFAP, specifically tailored for multi-face manipulation detection and localization. The framework incorporates two novel modules: the Forgery-aware Unified Predictor (FUP) Module and the Mixture-of-Noises Module (MNM)
  arXiv  Detail & Related papers  (2024-08-05T08:35:59Z)
• Masked Collaborative Contrast for Weakly Supervised Semantic Segmentation [22.74105261883464]
  Masked Collaborative Contrast (MCC) to highlight semantic regions in weakly supervised semantic segmentation. MCC adroitly draws inspiration from masked image modeling and contrastive learning to devise a novel framework that induces keys to contract toward semantic regions.
  arXiv  Detail & Related papers  (2023-05-15T09:46:28Z)
• Masked Frequency Modeling for Self-Supervised Visual Pre-Training [102.89756957704138]
  We present Masked Frequency Modeling (MFM), a unified frequency-domain-based approach for self-supervised pre-training of visual models. MFM first masks out a portion of frequency components of the input image and then predicts the missing frequencies on the frequency spectrum. For the first time, MFM demonstrates that, for both ViT and CNN, a simple non-Siamese framework can learn meaningful representations even using none of the following: (i) extra data, (ii) extra model, (iii) mask token.
  arXiv  Detail & Related papers  (2022-06-15T17:58:30Z)
• TBNet:Two-Stream Boundary-aware Network for Generic Image Manipulation Localization [49.521622399483846]
  We propose a novel end-to-end two-stream boundary-aware network (abbreviated as TBNet) for generic image manipulation localization. The proposed TBNet can significantly outperform state-of-the-art generic image manipulation localization methods in terms of both MCC and F1.
  arXiv  Detail & Related papers  (2021-08-10T08:22:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.