On the detection of synthetic images generated by diffusion models
- URL: http://arxiv.org/abs/2211.00680v1
- Date: Tue, 1 Nov 2022 18:10:55 GMT
- Title: On the detection of synthetic images generated by diffusion models
- Authors: Riccardo Corvi and Davide Cozzolino and Giada Zingarini and Giovanni
Poggi and Koki Nagano and Luisa Verdoliva
- Abstract summary: Methods based on diffusion models (DM) have been gaining the spotlight.
DM enables the creation of text-based visual content.
Malicious users can generate and distribute fake media perfectly adapted to their attacks.
- Score: 18.12766911229293
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Over the past decade, there has been tremendous progress in creating
synthetic media, mainly thanks to the development of powerful methods based on
generative adversarial networks (GAN). Very recently, methods based on
diffusion models (DM) have been gaining the spotlight. In addition to providing
an impressive level of photorealism, they enable the creation of text-based
visual content, opening up new and exciting opportunities in many different
application fields, from arts to video games. On the other hand, this property
is an additional asset in the hands of malicious users, who can generate and
distribute fake media perfectly adapted to their attacks, posing new challenges
to the media forensic community. With this work, we seek to understand how
difficult it is to distinguish synthetic images generated by diffusion models
from pristine ones and whether current state-of-the-art detectors are suitable
for the task. To this end, first we expose the forensics traces left by
diffusion models, then study how current detectors, developed for GAN-generated
images, perform on these new synthetic images, especially in challenging
social-networks scenarios involving image compression and resizing. Datasets
and code are available at github.com/grip-unina/DMimageDetection.
Related papers
- Are CLIP features all you need for Universal Synthetic Image Origin Attribution? [13.96698277726253]
We propose a framework that incorporates features from large pre-trained foundation models to perform Open-Set origin attribution of synthetic images.
We show that our method leads to remarkable attribution performance, even in the low-data regime.
arXiv Detail & Related papers (2024-08-17T09:54:21Z) - When Synthetic Traces Hide Real Content: Analysis of Stable Diffusion Image Laundering [18.039034362749504]
In recent years, methods for producing highly realistic synthetic images have significantly advanced.
It is possible to pass an image through SD autoencoders to reproduce a synthetic copy of the image with high realism and almost no visual artifacts.
This process, known as SD image laundering, can transform real images into lookalike synthetic ones and risks complicating forensic analysis for content authenticity verification.
arXiv Detail & Related papers (2024-07-15T14:01:35Z) - Coarse-to-Fine Latent Diffusion for Pose-Guided Person Image Synthesis [65.7968515029306]
We propose a novel Coarse-to-Fine Latent Diffusion (CFLD) method for Pose-Guided Person Image Synthesis (PGPIS)
A perception-refined decoder is designed to progressively refine a set of learnable queries and extract semantic understanding of person images as a coarse-grained prompt.
arXiv Detail & Related papers (2024-02-28T06:07:07Z) - Learned representation-guided diffusion models for large-image generation [58.192263311786824]
We introduce a novel approach that trains diffusion models conditioned on embeddings from self-supervised learning (SSL)
Our diffusion models successfully project these features back to high-quality histopathology and remote sensing images.
Augmenting real data by generating variations of real images improves downstream accuracy for patch-level and larger, image-scale classification tasks.
arXiv Detail & Related papers (2023-12-12T14:45:45Z) - Text-image guided Diffusion Model for generating Deepfake celebrity
interactions [50.37578424163951]
Diffusion models have recently demonstrated highly realistic visual content generation.
This paper devises and explores a novel method in that regard.
Our results show that with the devised scheme, it is possible to create fake visual content with alarming realism.
arXiv Detail & Related papers (2023-09-26T08:24:37Z) - Generalizable Synthetic Image Detection via Language-guided Contrastive
Learning [22.4158195581231]
malevolent use of synthetic images, such as the dissemination of fake news or the creation of fake profiles, raises significant concerns regarding the authenticity of images.
We propose a simple yet very effective synthetic image detection method via a language-guided contrastive learning and a new formulation of the detection problem.
It is shown that our proposed LanguAge-guided SynThEsis Detection (LASTED) model achieves much improved generalizability to unseen image generation models.
arXiv Detail & Related papers (2023-05-23T08:13:27Z) - Parents and Children: Distinguishing Multimodal DeepFakes from Natural Images [60.34381768479834]
Recent advancements in diffusion models have enabled the generation of realistic deepfakes from textual prompts in natural language.
We pioneer a systematic study on deepfake detection generated by state-of-the-art diffusion models.
arXiv Detail & Related papers (2023-04-02T10:25:09Z) - Deep Image Fingerprint: Towards Low Budget Synthetic Image Detection and Model Lineage Analysis [8.777277201807351]
We develop a new detection method for images that are indistinguishable from real ones.
Our method can detect images from a known generative model and enable us to establish relationships between fine-tuned generative models.
Our approach achieves comparable performance to state-of-the-art pre-trained detection methods on images generated by Stable Diffusion and Midversa.
arXiv Detail & Related papers (2023-03-19T20:31:38Z) - DIRE for Diffusion-Generated Image Detection [128.95822613047298]
We propose a novel representation called DIffusion Reconstruction Error (DIRE)
DIRE measures the error between an input image and its reconstruction counterpart by a pre-trained diffusion model.
It provides a hint that DIRE can serve as a bridge to distinguish generated and real images.
arXiv Detail & Related papers (2023-03-16T13:15:03Z) - Person Image Synthesis via Denoising Diffusion Model [116.34633988927429]
We show how denoising diffusion models can be applied for high-fidelity person image synthesis.
Our results on two large-scale benchmarks and a user study demonstrate the photorealism of our proposed approach under challenging scenarios.
arXiv Detail & Related papers (2022-11-22T18:59:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.