Improving Synthetically Generated Image Detection in Cross-Concept Settings

• URL: http://arxiv.org/abs/2304.12053v1
• Date: Mon, 24 Apr 2023 12:45:00 GMT
• Title: Improving Synthetically Generated Image Detection in Cross-Concept Settings
• Authors: Pantelis Dogoulis, Giorgos Kordopatis-Zilos, Ioannis Kompatsiaris, Symeon Papadopoulos
• Abstract summary: We focus on the challenge of generalizing across different concept classes, e.g., when training a detector on human faces. We propose an approach based on the premise that the robustness of the detector can be enhanced by training it on realistic synthetic images.
• Score: 20.21594285488186
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: New advancements for the detection of synthetic images are critical for fighting disinformation, as the capabilities of generative AI models continuously evolve and can lead to hyper-realistic synthetic imagery at unprecedented scale and speed. In this paper, we focus on the challenge of generalizing across different concept classes, e.g., when training a detector on human faces and testing on synthetic animal images - highlighting the ineffectiveness of existing approaches that randomly sample generated images to train their models. By contrast, we propose an approach based on the premise that the robustness of the detector can be enhanced by training it on realistic synthetic images that are selected based on their quality scores according to a probabilistic quality estimation model. We demonstrate the effectiveness of the proposed approach by conducting experiments with generated images from two seminal architectures, StyleGAN2 and Latent Diffusion, and using three different concepts for each, so as to measure the cross-concept generalization ability. Our results show that our quality-based sampling method leads to higher detection performance for nearly all concepts, improving the overall effectiveness of the synthetic image detectors.

Related papers

• Time Step Generating: A Universal Synthesized Deepfake Image Detector [0.4488895231267077]
  We propose a universal synthetic image detector Time Step Generating (TSG) TSG does not rely on pre-trained models' reconstructing ability, specific datasets, or sampling algorithms. We test the proposed TSG on the large-scale GenImage benchmark and it achieves significant improvements in both accuracy and generalizability.
  arXiv  Detail & Related papers  (2024-11-17T09:39:50Z)
• Bi-LORA: A Vision-Language Approach for Synthetic Image Detection [14.448350657613364]
  Deep image synthesis techniques, such as generative adversarial networks (GANs) and diffusion models (DMs) have ushered in an era of generating highly realistic images. This paper takes inspiration from the potent convergence capabilities between vision and language, coupled with the zero-shot nature of vision-language models (VLMs) We introduce an innovative method called Bi-LORA that leverages VLMs, combined with low-rank adaptation (LORA) tuning techniques, to enhance the precision of synthetic image detection for unseen model-generated images.
  arXiv  Detail & Related papers  (2024-04-02T13:54:22Z)
• Is Synthetic Image Useful for Transfer Learning? An Investigation into Data Generation, Volume, and Utilization [62.157627519792946]
  We introduce a novel framework called bridged transfer, which initially employs synthetic images for fine-tuning a pre-trained model to improve its transferability. We propose dataset style inversion strategy to improve the stylistic alignment between synthetic and real images. Our proposed methods are evaluated across 10 different datasets and 5 distinct models, demonstrating consistent improvements.
  arXiv  Detail & Related papers  (2024-03-28T22:25:05Z)
• Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detection [106.39544368711427]
  We study the problem of generalizable synthetic image detection, aiming to detect forgery images from diverse generative methods. We present a novel forgery-aware adaptive transformer approach, namely FatFormer. Our approach tuned on 4-class ProGAN data attains an average of 98% accuracy to unseen GANs, and surprisingly generalizes to unseen diffusion models with 95% accuracy.
  arXiv  Detail & Related papers  (2023-12-27T17:36:32Z)
• Generalizable Synthetic Image Detection via Language-guided Contrastive Learning [22.4158195581231]
  malevolent use of synthetic images, such as the dissemination of fake news or the creation of fake profiles, raises significant concerns regarding the authenticity of images. We propose a simple yet very effective synthetic image detection method via a language-guided contrastive learning and a new formulation of the detection problem. It is shown that our proposed LanguAge-guided SynThEsis Detection (LASTED) model achieves much improved generalizability to unseen image generation models.
  arXiv  Detail & Related papers  (2023-05-23T08:13:27Z)
• IRGen: Generative Modeling for Image Retrieval [82.62022344988993]
  In this paper, we present a novel methodology, reframing image retrieval as a variant of generative modeling. We develop our model, dubbed IRGen, to address the technical challenge of converting an image into a concise sequence of semantic units. Our model achieves state-of-the-art performance on three widely-used image retrieval benchmarks and two million-scale datasets.
  arXiv  Detail & Related papers  (2023-03-17T17:07:36Z)
• IMAGINE: Image Synthesis by Image-Guided Model Inversion [79.4691654458141]
  We introduce an inversion based method, denoted as IMAge-Guided model INvErsion (IMAGINE), to generate high-quality and diverse images. We leverage the knowledge of image semantics from a pre-trained classifier to achieve plausible generations. IMAGINE enables the synthesis procedure to simultaneously 1) enforce semantic specificity constraints during the synthesis, 2) produce realistic images without generator training, and 3) give users intuitive control over the generation process.
  arXiv  Detail & Related papers  (2021-04-13T02:00:24Z)
• Identity-Aware CycleGAN for Face Photo-Sketch Synthesis and Recognition [61.87842307164351]
  We first propose an Identity-Aware CycleGAN (IACycleGAN) model that applies a new perceptual loss to supervise the image generation network. It improves CycleGAN on photo-sketch synthesis by paying more attention to the synthesis of key facial regions, such as eyes and nose. We develop a mutual optimization procedure between the synthesis model and the recognition model, which iteratively synthesizes better images by IACycleGAN.
  arXiv  Detail & Related papers  (2021-03-30T01:30:08Z)
• Synthetic Sample Selection via Reinforcement Learning [8.099072894865802]
  We propose a reinforcement learning based synthetic sample selection method that learns to choose synthetic images containing reliable and informative features. In experiments on a cervical dataset and a lymph node dataset, the image classification performance is improved by 8.1% and 2.3%, respectively.
  arXiv  Detail & Related papers  (2020-08-26T01:34:19Z)
• Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wild [98.48284827503409]
  We develop a textitunified BIQA model and an approach of training it for both synthetic and realistic distortions. We employ the fidelity loss to optimize a deep neural network for BIQA over a large number of such image pairs. Experiments on six IQA databases show the promise of the learned method in blindly assessing image quality in the laboratory and wild.
  arXiv  Detail & Related papers  (2020-05-28T13:35:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.