Auto-Focus Contrastive Learning for Image Manipulation Detection

• URL: http://arxiv.org/abs/2211.10922v1
• Date: Sun, 20 Nov 2022 09:40:36 GMT
• Title: Auto-Focus Contrastive Learning for Image Manipulation Detection
• Authors: Wenyan Pan, Zhili Zhou, Guangcan Liu, Teng Huang, Hongyang Yan, Q.M. Jonathan Wu
• Abstract summary: We propose an Auto-Focus Contrastive Learning (AF-CL) network for image manipulation detection. It contains two main ideas, i.e., multi-scale view generation (MSVG) and trace relation modeling (TRM)
• Score: 28.332585163675617
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generally, current image manipulation detection models are simply built on manipulation traces. However, we argue that those models achieve sub-optimal detection performance as it tends to: 1) distinguish the manipulation traces from a lot of noisy information within the entire image, and 2) ignore the trace relations among the pixels of each manipulated region and its surroundings. To overcome these limitations, we propose an Auto-Focus Contrastive Learning (AF-CL) network for image manipulation detection. It contains two main ideas, i.e., multi-scale view generation (MSVG) and trace relation modeling (TRM). Specifically, MSVG aims to generate a pair of views, each of which contains the manipulated region and its surroundings at a different scale, while TRM plays a role in modeling the trace relations among the pixels of each manipulated region and its surroundings for learning the discriminative representation. After learning the AF-CL network by minimizing the distance between the representations of corresponding views, the learned network is able to automatically focus on the manipulated region and its surroundings and sufficiently explore their trace relations for accurate manipulation detection. Extensive experiments demonstrate that, compared to the state-of-the-arts, AF-CL provides significant performance improvements, i.e., up to 2.5%, 7.5%, and 0.8% F1 score, on CAISA, NIST, and Coverage datasets, respectively.

Related papers

• MLMT-CNN for Object Detection and Segmentation in Multi-layer and Multi-spectral Images [4.2623421577291225]
  We present a multi-task deep learning framework that exploits the dependencies between image bands to produce 3D AR localisation. Our framework achieves an average of 0.72 IoU (segmentation) and 0.90 F1 score (detection) across all modalities.
  arXiv  Detail & Related papers  (2024-07-19T17:21:53Z)
• Learning 1D Causal Visual Representation with De-focus Attention Networks [108.72931590504406]
  This paper explores the feasibility of representing images using 1D causal modeling. We propose De-focus Attention Networks, which employ learnable bandpass filters to create varied attention patterns.
  arXiv  Detail & Related papers  (2024-06-06T17:59:56Z)
• Pixel-Inconsistency Modeling for Image Manipulation Localization [59.968362815126326]
  Digital image forensics plays a crucial role in image authentication and manipulation localization. This paper presents a generalized and robust manipulation localization model through the analysis of pixel inconsistency artifacts. Experiments show that our method successfully extracts inherent pixel-inconsistency forgery fingerprints.
  arXiv  Detail & Related papers  (2023-09-30T02:54:51Z)
• Towards Generic Image Manipulation Detection with Weakly-Supervised Self-Consistency Learning [49.43362803584032]
  We propose weakly-supervised image manipulation detection. Such a setting can leverage more training images and has the potential to adapt quickly to new manipulation techniques. Two consistency properties are learned: multi-source consistency (MSC) and inter-patch consistency (IPC)
  arXiv  Detail & Related papers  (2023-09-03T19:19:56Z)
• Towards Effective Image Manipulation Detection with Proposal Contrastive Learning [61.5469708038966]
  We propose Proposal Contrastive Learning (PCL) for effective image manipulation detection. Our PCL consists of a two-stream architecture by extracting two types of global features from RGB and noise views respectively. Our PCL can be easily adapted to unlabeled data in practice, which can reduce manual labeling costs and promote more generalizable features.
  arXiv  Detail & Related papers  (2022-10-16T13:30:13Z)
• Noise and Edge Based Dual Branch Image Manipulation Detection [9.400611271697302]
  In this paper, the noise image extracted by the improved constrained convolution is used as the input of the model. The dual-branch network, consisting of a high-resolution branch and a context branch, is used to capture the traces of artifacts as much as possible. A specially designed manipulation edge detection module is constructed based on the dual-branch network to identify these artifacts better.
  arXiv  Detail & Related papers  (2022-07-02T03:28:51Z)
• Learning Hierarchical Graph Representation for Image Manipulation Detection [50.04902159383709]
  The objective of image manipulation detection is to identify and locate the manipulated regions in the images. Recent approaches mostly adopt the sophisticated Convolutional Neural Networks (CNNs) to capture the tampering artifacts left in the images. We propose a hierarchical Graph Convolutional Network (HGCN-Net), which consists of two parallel branches.
  arXiv  Detail & Related papers  (2022-01-15T01:54:25Z)
• Image Manipulation Detection by Multi-View Multi-Scale Supervision [11.319080833880307]
  Key challenge of image manipulation detection is how to learn generalizable features that are sensitive to manipulations in novel data. In this paper we address both aspects by multi-view feature learning and multi-scale supervision. Our thoughts are realized by a new network which we term MVSS-Net.
  arXiv  Detail & Related papers  (2021-04-14T13:05:58Z)
• DetCo: Unsupervised Contrastive Learning for Object Detection [64.22416613061888]
  Unsupervised contrastive learning achieves great success in learning image representations with CNN. We present a novel contrastive learning approach, named DetCo, which fully explores the contrasts between global image and local image patches. DetCo consistently outperforms supervised method by 1.6/1.2/1.0 AP on Mask RCNN-C4/FPN/RetinaNet with 1x schedule.
  arXiv  Detail & Related papers  (2021-02-09T12:47:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.