Transfer Learning of Real Image Features with Soft Contrastive Loss for Fake Image Detection

• URL: http://arxiv.org/abs/2403.16513v2
• Date: Mon, 03 Mar 2025 16:12:09 GMT
• Title: Transfer Learning of Real Image Features with Soft Contrastive Loss for Fake Image Detection
• Authors: Ziyou Liang, Weifeng Liu, Run Wang, Mengjie Wu, Boheng Li, Yuyang Zhang, Lina Wang, Xinyi Yang,
• Abstract summary: We introduce a self-supervised feature mapping process for natural trace extraction and develop a transfer learning based on soft contrastive loss.<n>This motivates the detector to make decisions based on the proximity of images to the natural traces.<n>Our proposed method gives 96.2% mAP significantly outperforms the baselines.
• Score: 15.89388721108276
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the last few years, the artifact patterns in fake images synthesized by different generative models have been inconsistent, leading to the failure of previous research that relied on spotting subtle differences between real and fake. In our preliminary experiments, we find that the artifacts in fake images always change with the development of the generative model, while natural images exhibit stable statistical properties. In this paper, we employ natural traces shared only by real images as an additional target for a classifier. Specifically, we introduce a self-supervised feature mapping process for natural trace extraction and develop a transfer learning based on soft contrastive loss to bring them closer to real images and further away from fake ones. This motivates the detector to make decisions based on the proximity of images to the natural traces. To conduct a comprehensive experiment, we built a high-quality and diverse dataset that includes generative models comprising GANs and diffusion models, to evaluate the effectiveness in generalizing unknown forgery techniques and robustness in surviving different transformations. Experimental results show that our proposed method gives 96.2% mAP significantly outperforms the baselines. Extensive experiments conducted on popular commercial platforms reveal that our proposed method achieves an accuracy exceeding 78.4%, underscoring its practicality for real-world application deployment.

Related papers

• Hiding Images in Diffusion Models by Editing Learned Score Functions [27.130542925771692]
  Current methods exhibit limitations in achieving high extraction accuracy, model fidelity, and hiding efficiency. We describe a simple yet effective approach that embeds images at specific timesteps in the reverse diffusion process by editing the learned score functions. We also introduce a parameter-efficient fine-tuning method that combines gradient-based parameter selection with low-rank adaptation to enhance model fidelity and hiding efficiency.
  arXiv  Detail & Related papers  (2025-03-24T09:04:25Z)
• Explainable Synthetic Image Detection through Diffusion Timestep Ensembling [30.298198387824275]
  Recent advances in diffusion models have enabled the creation of deceptively real images. Recent advances in diffusion models have enabled the creation of deceptively real images, posing significant security risks when misused.
  arXiv  Detail & Related papers  (2025-03-08T13:04:20Z)
• DiffDoctor: Diagnosing Image Diffusion Models Before Treating [57.82359018425674]
  We propose DiffDoctor, a two-stage pipeline to assist image diffusion models in generating fewer artifacts. We collect a dataset of over 1M flawed synthesized images and set up an efficient human-in-the-loop annotation process. The learned artifact detector is then involved in the second stage to optimize the diffusion model by providing pixel-level feedback.
  arXiv  Detail & Related papers  (2025-01-21T18:56:41Z)
• Understanding and Improving Training-Free AI-Generated Image Detections with Vision Foundation Models [68.90917438865078]
  Deepfake techniques for facial synthesis and editing pose serious risks for generative models. In this paper, we investigate how detection performance varies across model backbones, types, and datasets. We introduce Contrastive Blur, which enhances performance on facial images, and MINDER, which addresses noise type bias, balancing performance across domains.
  arXiv  Detail & Related papers  (2024-11-28T13:04:45Z)
• Generating Realistic X-ray Scattering Images Using Stable Diffusion and Human-in-the-loop Annotations [42.47750355293256]
  We fine-tuned a foundational stable diffusion model to generate new scientific images from given prompts. Some of the generated images exhibit significant unrealistic artifacts, commonly known as "hallucinations" We trained various computer vision models on a dataset composed of 60% human-approved generated images and 40% experimental images to detect unrealistic images.
  arXiv  Detail & Related papers  (2024-08-22T20:23:04Z)
• Contrasting Deepfakes Diffusion via Contrastive Learning and Global-Local Similarities [88.398085358514]
  Contrastive Deepfake Embeddings (CoDE) is a novel embedding space specifically designed for deepfake detection. CoDE is trained via contrastive learning by additionally enforcing global-local similarities.
  arXiv  Detail & Related papers  (2024-07-29T18:00:10Z)
• Diff-Mosaic: Augmenting Realistic Representations in Infrared Small Target Detection via Diffusion Prior [63.64088590653005]
  We propose Diff-Mosaic, a data augmentation method based on the diffusion model. We introduce an enhancement network called Pixel-Prior, which generates highly coordinated and realistic Mosaic images. In the second stage, we propose an image enhancement strategy named Diff-Prior. This strategy utilizes diffusion priors to model images in the real-world scene.
  arXiv  Detail & Related papers  (2024-06-02T06:23:05Z)
• Unveiling the Truth: Exploring Human Gaze Patterns in Fake Images [34.02058539403381]
  We leverage human semantic knowledge to investigate the possibility of being included in frameworks of fake image detection. A preliminary statistical analysis is conducted to explore the distinctive patterns in how humans perceive genuine and altered images.
  arXiv  Detail & Related papers  (2024-03-13T19:56:30Z)
• Detecting Generated Images by Real Images Only [64.12501227493765]
  Existing generated image detection methods detect visual artifacts in generated images or learn discriminative features from both real and generated images by massive training. This paper approaches the generated image detection problem from a new perspective: Start from real images. By finding the commonality of real images and mapping them to a dense subspace in feature space, the goal is that generated images, regardless of their generative model, are then projected outside the subspace.
  arXiv  Detail & Related papers  (2023-11-02T03:09:37Z)
• Perceptual Artifacts Localization for Image Synthesis Tasks [59.638307505334076]
  We introduce a novel dataset comprising 10,168 generated images, each annotated with per-pixel perceptual artifact labels. A segmentation model, trained on our proposed dataset, effectively localizes artifacts across a range of tasks. We propose an innovative zoom-in inpainting pipeline that seamlessly rectifies perceptual artifacts in the generated images.
  arXiv  Detail & Related papers  (2023-10-09T10:22:08Z)
• On quantifying and improving realism of images generated with diffusion [50.37578424163951]
  We propose a metric, called Image Realism Score (IRS), computed from five statistical measures of a given image. IRS is easily usable as a measure to classify a given image as real or fake. We experimentally establish the model- and data-agnostic nature of the proposed IRS by successfully detecting fake images generated by Stable Diffusion Model (SDM), Dalle2, Midjourney and BigGAN. Our efforts have also led to Gen-100 dataset, which provides 1,000 samples for 100 classes generated by four high-quality models.
  arXiv  Detail & Related papers  (2023-09-26T08:32:55Z)
• Parents and Children: Distinguishing Multimodal DeepFakes from Natural Images [60.34381768479834]
  Recent advancements in diffusion models have enabled the generation of realistic deepfakes from textual prompts in natural language. We pioneer a systematic study on deepfake detection generated by state-of-the-art diffusion models.
  arXiv  Detail & Related papers  (2023-04-02T10:25:09Z)
• Image Completion via Inference in Deep Generative Models [16.99337751292915]
  We consider image completion from the perspective of amortized inference in an image generative model. We demonstrate superior sample quality and diversity compared to prior art on the CIFAR-10 and FFHQ-256 datasets.
  arXiv  Detail & Related papers  (2021-02-24T02:59:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.