Weakly-supervised Localization of Manipulated Image Regions Using Multi-resolution Learned Features

• URL: http://arxiv.org/abs/2505.23586v1
• Date: Thu, 29 May 2025 15:58:29 GMT
• Title: Weakly-supervised Localization of Manipulated Image Regions Using Multi-resolution Learned Features
• Authors: Ziyong Wang, Charith Abhayaratne,
• Abstract summary: Current deep learning-based manipulation detection methods excel in achieving high image-level classification accuracy.<n>The absence of pixel-wise annotations in real-world scenarios limits the existing fully-supervised manipulation localization techniques.<n>We propose a novel weakly-supervised approach that integrates activation maps generated by image-level manipulation detection networks with segmentation maps from pre-trained models.
• Score: 4.83420384410068
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The explosive growth of digital images and the widespread availability of image editing tools have made image manipulation detection an increasingly critical challenge. Current deep learning-based manipulation detection methods excel in achieving high image-level classification accuracy, they often fall short in terms of interpretability and localization of manipulated regions. Additionally, the absence of pixel-wise annotations in real-world scenarios limits the existing fully-supervised manipulation localization techniques. To address these challenges, we propose a novel weakly-supervised approach that integrates activation maps generated by image-level manipulation detection networks with segmentation maps from pre-trained models. Specifically, we build on our previous image-level work named WCBnet to produce multi-view feature maps which are subsequently fused for coarse localization. These coarse maps are then refined using detailed segmented regional information provided by pre-trained segmentation models (such as DeepLab, SegmentAnything and PSPnet), with Bayesian inference employed to enhance the manipulation localization. Experimental results demonstrate the effectiveness of our approach, highlighting the feasibility to localize image manipulations without relying on pixel-level labels.

Related papers

• Context-Aware Weakly Supervised Image Manipulation Localization with SAM Refinement [52.15627062770557]
  Malicious image manipulation poses societal risks, increasing the importance of effective image manipulation detection methods.<n>Recent approaches in image manipulation detection have largely been driven by fully supervised approaches.<n>We present a novel weakly supervised framework based on a dual-branch Transformer-CNN architecture.
  arXiv  Detail & Related papers  (2025-03-26T07:35:09Z)
• Weakly-supervised deepfake localization in diffusion-generated images [4.548755617115687]
  We propose a weakly-supervised localization problem based on the Xception network as the backbone architecture. We show that the best performing detection method (based on local scores) is less sensitive to the looser supervision than to the mismatch in terms of dataset or generator.
  arXiv  Detail & Related papers  (2023-11-08T10:27:36Z)
• Pixel-Inconsistency Modeling for Image Manipulation Localization [59.968362815126326]
  Digital image forensics plays a crucial role in image authentication and manipulation localization. This paper presents a generalized and robust manipulation localization model through the analysis of pixel inconsistency artifacts. Experiments show that our method successfully extracts inherent pixel-inconsistency forgery fingerprints.
  arXiv  Detail & Related papers  (2023-09-30T02:54:51Z)
• Towards Effective Image Manipulation Detection with Proposal Contrastive Learning [61.5469708038966]
  We propose Proposal Contrastive Learning (PCL) for effective image manipulation detection. Our PCL consists of a two-stream architecture by extracting two types of global features from RGB and noise views respectively. Our PCL can be easily adapted to unlabeled data in practice, which can reduce manual labeling costs and promote more generalizable features.
  arXiv  Detail & Related papers  (2022-10-16T13:30:13Z)
• ObjectFormer for Image Manipulation Detection and Localization [118.89882740099137]
  We propose ObjectFormer to detect and localize image manipulations. We extract high-frequency features of the images and combine them with RGB features as multimodal patch embeddings. We conduct extensive experiments on various datasets and the results verify the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-03-28T12:27:34Z)
• Cross-Image Region Mining with Region Prototypical Network for Weakly Supervised Segmentation [45.39679291105364]
  We propose a region network RPNet to explore the cross-image object diversity of the training set. Similar object parts across images are identified via region feature comparison. Experiments show that the proposed method generates more complete and accurate pseudo object masks.
  arXiv  Detail & Related papers  (2021-08-17T02:51:02Z)
• Detect and Locate: A Face Anti-Manipulation Approach with Semantic and Noise-level Supervision [67.73180660609844]
  We propose a conceptually simple but effective method to efficiently detect forged faces in an image. The proposed scheme relies on a segmentation map that delivers meaningful high-level semantic information clues about the image. The proposed model achieves state-of-the-art detection accuracy and remarkable localization performance.
  arXiv  Detail & Related papers  (2021-07-13T02:59:31Z)
• Unsupervised Metric Relocalization Using Transform Consistency Loss [66.19479868638925]
  Training networks to perform metric relocalization traditionally requires accurate image correspondences. We propose a self-supervised solution, which exploits a key insight: localizing a query image within a map should yield the same absolute pose, regardless of the reference image used for registration. We evaluate our framework on synthetic and real-world data, showing our approach outperforms other supervised methods when a limited amount of ground-truth information is available.
  arXiv  Detail & Related papers  (2020-11-01T19:24:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.