Noise Doesn't Lie: Towards Universal Detection of Deep Inpainting

• URL: http://arxiv.org/abs/2106.01532v1
• Date: Thu, 3 Jun 2021 01:29:29 GMT
• Title: Noise Doesn't Lie: Towards Universal Detection of Deep Inpainting
• Authors: Ang Li, Qiuhong Ke, Xingjun Ma, Haiqin Weng, Zhiyuan Zong, Feng Xue, Rui Zhang
• Abstract summary: We make the first attempt towards universal detection of deep inpainting, where the detection network can generalize well. Our approach outperforms existing detection methods by a large margin and generalizes well to unseen deep inpainting techniques.
• Score: 42.189768203036394
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep image inpainting aims to restore damaged or missing regions in an image with realistic contents. While having a wide range of applications such as object removal and image recovery, deep inpainting techniques also have the risk of being manipulated for image forgery. A promising countermeasure against such forgeries is deep inpainting detection, which aims to locate the inpainted regions in an image. In this paper, we make the first attempt towards universal detection of deep inpainting, where the detection network can generalize well when detecting different deep inpainting methods. To this end, we first propose a novel data generation approach to generate a universal training dataset, which imitates the noise discrepancies exist in real versus inpainted image contents to train universal detectors. We then design a Noise-Image Cross-fusion Network (NIX-Net) to effectively exploit the discriminative information contained in both the images and their noise patterns. We empirically show, on multiple benchmark datasets, that our approach outperforms existing detection methods by a large margin and generalize well to unseen deep inpainting techniques. Our universal training dataset can also significantly boost the generalizability of existing detection methods.

Related papers

• Dense Feature Interaction Network for Image Inpainting Localization [28.028361409524457]
  Inpainting can be used to conceal or alter image contents in malicious manipulation of images. Existing methods mostly rely on a basic encoder-decoder structure, which often results in a high number of false positives. In this paper, we describe a new method for inpainting detection based on a Dense Feature Interaction Network (DeFI-Net)
  arXiv  Detail & Related papers  (2024-08-05T02:35:13Z)
• Contrasting Deepfakes Diffusion via Contrastive Learning and Global-Local Similarities [88.398085358514]
  Contrastive Deepfake Embeddings (CoDE) is a novel embedding space specifically designed for deepfake detection. CoDE is trained via contrastive learning by additionally enforcing global-local similarities.
  arXiv  Detail & Related papers  (2024-07-29T18:00:10Z)
• Detecting Generated Images by Real Images Only [64.12501227493765]
  Existing generated image detection methods detect visual artifacts in generated images or learn discriminative features from both real and generated images by massive training. This paper approaches the generated image detection problem from a new perspective: Start from real images. By finding the commonality of real images and mapping them to a dense subspace in feature space, the goal is that generated images, regardless of their generative model, are then projected outside the subspace.
  arXiv  Detail & Related papers  (2023-11-02T03:09:37Z)
• Masked Image Training for Generalizable Deep Image Denoising [53.03126421917465]
  We present a novel approach to enhance the generalization performance of denoising networks. Our method involves masking random pixels of the input image and reconstructing the missing information during training. Our approach exhibits better generalization ability than other deep learning models and is directly applicable to real-world scenarios.
  arXiv  Detail & Related papers  (2023-03-23T09:33:44Z)
• TruFor: Leveraging all-round clues for trustworthy image forgery detection and localization [17.270110456445806]
  TruFor is a forensic framework that can be applied to a large variety of image manipulation methods. We rely on the extraction of both high-level and low-level traces through a transformer-based fusion architecture. Our method is able to reliably detect and localize both cheapfakes and deepfakes manipulations outperforming state-of-the-art works.
  arXiv  Detail & Related papers  (2022-12-21T11:49:43Z)
• Inpainting Transformer for Anomaly Detection [0.0]
  Inpainting Transformer (InTra) is trained to inpaint covered patches in a large sequence of image patches. InTra achieves better than state-of-the-art results on the MVTec AD dataset for detection and localization.
  arXiv  Detail & Related papers  (2021-04-28T17:27:44Z)
• GIID-Net: Generalizable Image Inpainting Detection via Neural Architecture Search and Attention [19.599993572921065]
  malicious use of advanced image inpainting tools has led to increasing threats to the reliability of image data. To fight against the inpainting forgeries, we propose a novel end-to-end Generalizable Image Inpainting Detection Network (GIID-Net) The proposed GIID-Net consists of three sub-blocks: the enhancement block, the extraction block and the decision block.
  arXiv  Detail & Related papers  (2021-01-19T02:29:40Z)
• Image inpainting using frequency domain priors [35.54138025375951]
  We present a novel image inpainting technique using frequency domain information. We evaluate our proposed method on the publicly available datasets CelebA, Paris Streetview, and DTD texture dataset.
  arXiv  Detail & Related papers  (2020-12-03T11:08:13Z)
• Free-Form Image Inpainting via Contrastive Attention Network [64.05544199212831]
  In image inpainting tasks, masks with any shapes can appear anywhere in images which form complex patterns. It is difficult for encoders to capture such powerful representations under this complex situation. We propose a self-supervised Siamese inference network to improve the robustness and generalization.
  arXiv  Detail & Related papers  (2020-10-29T14:46:05Z)
• High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling [122.06593036862611]
  Existing image inpainting methods often produce artifacts when dealing with large holes in real applications. We propose an iterative inpainting method with a feedback mechanism. Experiments show that our method significantly outperforms existing methods in both quantitative and qualitative evaluations.
  arXiv  Detail & Related papers  (2020-05-24T13:23:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.