Rethinking Image Forgery Detection via Contrastive Learning and Unsupervised Clustering

• URL: http://arxiv.org/abs/2308.09307v1
• Date: Fri, 18 Aug 2023 05:05:30 GMT
• Title: Rethinking Image Forgery Detection via Contrastive Learning and Unsupervised Clustering
• Authors: Haiwei Wu and Yiming Chen and Jiantao Zhou
• Abstract summary: We propose FOrensic ContrAstive cLustering (FOCAL) method for image forgery detection. FOCAL is based on contrastive learning and unsupervised clustering. Results show FOCAL significantly outperforms state-of-the-art competing algorithms.
• Score: 26.923409536155166
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Image forgery detection aims to detect and locate forged regions in an image. Most existing forgery detection algorithms formulate classification problems to classify pixels into forged or pristine. However, the definition of forged and pristine pixels is only relative within one single image, e.g., a forged region in image A is actually a pristine one in its source image B (splicing forgery). Such a relative definition has been severely overlooked by existing methods, which unnecessarily mix forged (pristine) regions across different images into the same category. To resolve this dilemma, we propose the FOrensic ContrAstive cLustering (FOCAL) method, a novel, simple yet very effective paradigm based on contrastive learning and unsupervised clustering for the image forgery detection. Specifically, FOCAL 1) utilizes pixel-level contrastive learning to supervise the high-level forensic feature extraction in an image-by-image manner, explicitly reflecting the above relative definition; 2) employs an on-the-fly unsupervised clustering algorithm (instead of a trained one) to cluster the learned features into forged/pristine categories, further suppressing the cross-image influence from training data; and 3) allows to further boost the detection performance via simple feature-level concatenation without the need of retraining. Extensive experimental results over six public testing datasets demonstrate that our proposed FOCAL significantly outperforms the state-of-the-art competing algorithms by big margins: +24.3% on Coverage, +18.6% on Columbia, +17.5% on FF++, +14.2% on MISD, +13.5% on CASIA and +10.3% on NIST in terms of IoU. The paradigm of FOCAL could bring fresh insights and serve as a novel benchmark for the image forgery detection task. The code is available at https://github.com/HighwayWu/FOCAL.

Related papers

• Annotation Cost-Efficient Active Learning for Deep Metric Learning Driven Remote Sensing Image Retrieval [3.2109665109975696]
  ANNEAL aims to create a small but informative training set made up of similar and dissimilar image pairs. The informativeness of image pairs is evaluated by combining uncertainty and diversity criteria. This way of annotating images significantly reduces the annotation cost compared to annotating images with land-use land-cover class labels.
  arXiv  Detail & Related papers  (2024-06-14T15:08:04Z)
• Learning to Rank Patches for Unbiased Image Redundancy Reduction [80.93989115541966]
  Images suffer from heavy spatial redundancy because pixels in neighboring regions are spatially correlated. Existing approaches strive to overcome this limitation by reducing less meaningful image regions. We propose a self-supervised framework for image redundancy reduction called Learning to Rank Patches.
  arXiv  Detail & Related papers  (2024-03-31T13:12:41Z)
• MuSc: Zero-Shot Industrial Anomaly Classification and Segmentation with Mutual Scoring of the Unlabeled Images [12.48347948647802]
  We study zero-shot anomaly classification (AC) and segmentation (AS) in industrial vision. We leverage a discriminative characteristic to design a novel zero-shot AC/AS method by Mutual Scoring (MuSc) of the unlabeled images. We present an optimization approach named Re-scoring with Constrained Image-level Neighborhood (RsCIN) for image-level anomaly classification.
  arXiv  Detail & Related papers  (2024-01-30T05:16:52Z)
• Multilevel Saliency-Guided Self-Supervised Learning for Image Anomaly Detection [15.212031255539022]
  Anomaly detection (AD) is a fundamental task in computer vision. We propose CutSwap, which leverages saliency guidance to incorporate semantic cues for augmentation. CutSwap achieves state-of-the-art AD performance on two mainstream AD benchmark datasets.
  arXiv  Detail & Related papers  (2023-11-30T08:03:53Z)
• Fine-grained Recognition with Learnable Semantic Data Augmentation [68.48892326854494]
  Fine-grained image recognition is a longstanding computer vision challenge. We propose diversifying the training data at the feature-level to alleviate the discriminative region loss problem. Our method significantly improves the generalization performance on several popular classification networks.
  arXiv  Detail & Related papers  (2023-09-01T11:15:50Z)
• MisMatch: Calibrated Segmentation via Consistency on Differential Morphological Feature Perturbations with Limited Labels [5.500466607182699]
  Semi-supervised learning is a promising paradigm to address the issue of label scarcity in medical imaging. MisMatch is a semi-supervised segmentation framework based on the consistency between paired predictions.
  arXiv  Detail & Related papers  (2021-10-23T09:22:41Z)
• DetCo: Unsupervised Contrastive Learning for Object Detection [64.22416613061888]
  Unsupervised contrastive learning achieves great success in learning image representations with CNN. We present a novel contrastive learning approach, named DetCo, which fully explores the contrasts between global image and local image patches. DetCo consistently outperforms supervised method by 1.6/1.2/1.0 AP on Mask RCNN-C4/FPN/RetinaNet with 1x schedule.
  arXiv  Detail & Related papers  (2021-02-09T12:47:20Z)
• Dense Contrastive Learning for Self-Supervised Visual Pre-Training [102.15325936477362]
  We present dense contrastive learning, which implements self-supervised learning by optimizing a pairwise contrastive (dis)similarity loss at the pixel level between two views of input images. Compared to the baseline method MoCo-v2, our method introduces negligible computation overhead (only 1% slower)
  arXiv  Detail & Related papers  (2020-11-18T08:42:32Z)
• SCAN: Learning to Classify Images without Labels [73.69513783788622]
  We advocate a two-step approach where feature learning and clustering are decoupled. A self-supervised task from representation learning is employed to obtain semantically meaningful features. We obtain promising results on ImageNet, and outperform several semi-supervised learning methods in the low-data regime.
  arXiv  Detail & Related papers  (2020-05-25T18:12:33Z)
• High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification [84.43394420267794]
  We propose a novel framework by learning high-order relation and topology information for discriminative features and robust alignment. Our framework significantly outperforms state-of-the-art by6.5%mAP scores on Occluded-Duke dataset.
  arXiv  Detail & Related papers  (2020-03-18T12:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.