Semantic Manipulation Localization

• URL: http://arxiv.org/abs/2604.10132v1
• Date: Sat, 11 Apr 2026 09:53:09 GMT
• Title: Semantic Manipulation Localization
• Abstract summary: We introduce semantic manipulation, a new task that focuses on localizing subtle semantic edits that significantly change image interpretation.<n>Based on this task, we propose TRACE, an end-to-end framework that models semantic sensitivity through three components: semantic anchoring, perturbation sensing, and semantic-constrained reasoning.<n>Extensive experiments show that TRACE consistently outperforms existing IML methods on our benchmark and produces more complete, compact, and semantically coherent localization results.
• Score: 18.942761820082705
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Image Manipulation Localization (IML) aims to identify edited regions in an image. However, with the increasing use of modern image editing and generative models, many manipulations no longer exhibit obvious low-level artifacts. Instead, they often involve subtle but meaning-altering edits to an object's attributes, state, or relationships while remaining highly consistent with the surrounding content. This makes conventional IML methods less effective because they mainly rely on artifact detection rather than semantic sensitivity. To address this issue, we introduce Semantic Manipulation Localization (SML), a new task that focuses on localizing subtle semantic edits that significantly change image interpretation. We further construct a dedicated fine-grained benchmark for SML using a semantics-driven manipulation pipeline with pixel-level annotations. Based on this task, we propose TRACE (Targeted Reasoning of Attributed Cognitive Edits), an end-to-end framework that models semantic sensitivity through three progressively coupled components: semantic anchoring, semantic perturbation sensing, and semantic-constrained reasoning. Specifically, TRACE first identifies semantically meaningful regions that support image understanding, then injects perturbation-sensitive frequency cues to capture subtle edits under strong visual consistency, and finally verifies candidate regions through joint reasoning over semantic content and semantic scope. Extensive experiments show that TRACE consistently outperforms existing IML methods on our benchmark and produces more complete, compact, and semantically coherent localization results. These results demonstrate the necessity of moving beyond artifact-based localization and provide a new direction for image forensics in complex semantic editing scenarios.

Related papers

• Robust Grounding with MLLMs Against Occlusion and Small Objects via Language-Guided Semantic Cues [58.21371989991432]
  Crowded scenes entail visual challenges, which impair object semantics and degrade grounding performance.<n>We propose a novel method that overcomes such constraints by leveraging Language-Guided Semantic Cues (LGSCs)
  arXiv  Detail & Related papers  (2026-04-27T04:42:03Z)
• DQE-CIR: Distinctive Query Embeddings through Learnable Attribute Weights and Target Relative Negative Sampling in Composed Image Retrieval [53.482391830683014]
  Composed image retrieval (CIR) addresses the task of retrieving a target image by jointly interpreting a reference image and a modification text that specifies the intended change.<n>Most existing methods are still built upon contrastive learning frameworks that treat the ground truth image as the only positive instance and all remaining images as negatives.<n>We propose distinctive query embeddings through learnable attribute weights and target relative negative sampling.
  arXiv  Detail & Related papers  (2026-03-04T13:17:44Z)
• Seg4Diff: Unveiling Open-Vocabulary Segmentation in Text-to-Image Diffusion Transformers [56.76198904599581]
  Text-to-image diffusion models excel at translating language prompts into implicitly grounding concepts through their cross-modal attention mechanisms.<n>Recent multi-modal diffusion transformers extend this by introducing joint self-attentiond image and text tokens, enabling richer and more scalable cross-modal alignment.<n>We introduce Seg4Diff, a systematic framework for analyzing the attention structures of MM-DiT, with a focus on how specific layers propagate semantic information from text to image.
  arXiv  Detail & Related papers  (2025-09-22T17:59:54Z)
• Weakly-Supervised Image Forgery Localization via Vision-Language Collaborative Reasoning Framework [16.961220047066792]
  ViLaCo is a vision-language collaborative reasoning framework that introduces auxiliary semantic supervision distilled from pre-trained vision-language models.<n>ViLaCo substantially outperforms existing WSIFL methods, achieving state-of-the-art performance in both detection and localization accuracy.
  arXiv  Detail & Related papers  (2025-08-02T12:14:29Z)
• Context-Aware Weakly Supervised Image Manipulation Localization with SAM Refinement [52.15627062770557]
  Malicious image manipulation poses societal risks, increasing the importance of effective image manipulation detection methods.<n>Recent approaches in image manipulation detection have largely been driven by fully supervised approaches.<n>We present a novel weakly supervised framework based on a dual-branch Transformer-CNN architecture.
  arXiv  Detail & Related papers  (2025-03-26T07:35:09Z)
• EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Models [52.3015009878545]
  We develop an image segmentor capable of generating fine-grained segmentation maps without any additional training. Our framework identifies semantic correspondences between image pixels and spatial locations of low-dimensional feature maps. In extensive experiments, the produced segmentation maps are demonstrated to be well delineated and capture detailed parts of the images.
  arXiv  Detail & Related papers  (2024-01-22T07:34:06Z)
• SemST: Semantically Consistent Multi-Scale Image Translation via Structure-Texture Alignment [32.41465452443824]
  Unsupervised image-to-image (I2I) translation learns cross-domain image mapping that transfers input from the source domain to output in the target domain. Different semantic statistics in source and target domains result in content discrepancy known as semantic distortion. A novel I2I method that maintains semantic consistency in translation is proposed and named SemST in this work.
  arXiv  Detail & Related papers  (2023-10-08T03:44:58Z)
• LAW-Diffusion: Complex Scene Generation by Diffusion with Layouts [107.11267074981905]
  We propose a semantically controllable layout-AWare diffusion model, termed LAW-Diffusion. We show that LAW-Diffusion yields the state-of-the-art generative performance, especially with coherent object relations.
  arXiv  Detail & Related papers  (2023-08-13T08:06:18Z)
• Weakly Supervised Object Localization via Transformer with Implicit Spatial Calibration [20.322494442959762]
  Weakly Supervised Object Localization (WSOL) has attracted much attention because of its low annotation cost in real applications. We introduce a simple yet effective Spatial Module (SCM) for accurate WSOL, incorporating semantic similarities of patch tokens and their spatial relationships into a unified diffusion model. SCM is designed as an external module of Transformer, and can be removed during inference to reduce the computation cost.
  arXiv  Detail & Related papers  (2022-07-21T12:37:15Z)
• Situational Perception Guided Image Matting [16.1897179939677]
  We propose a Situational Perception Guided Image Matting (SPG-IM) method that mitigates subjective bias of matting annotations. SPG-IM can better associate inter-objects and object-to-environment saliency, and compensate the subjective nature of image matting.
  arXiv  Detail & Related papers  (2022-04-20T07:35:51Z)
• Semantically Adaptive Image-to-image Translation for Domain Adaptation of Semantic Segmentation [1.8275108630751844]
  We address the problem of domain adaptation for semantic segmentation of street scenes. Many state-of-the-art approaches focus on translating the source image while imposing that the result should be semantically consistent with the input. We advocate that the image semantics can also be exploited to guide the translation algorithm.
  arXiv  Detail & Related papers  (2020-09-02T16:16:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.