Towards Discovery and Attribution of Open-world GAN Generated Images
- URL: http://arxiv.org/abs/2105.04580v1
- Date: Mon, 10 May 2021 18:00:13 GMT
- Title: Towards Discovery and Attribution of Open-world GAN Generated Images
- Authors: Sharath Girish, Saksham Suri, Saketh Rambhatla, Abhinav Shrivastava
- Abstract summary: We present an iterative algorithm for discovering images generated from previously unseen GANs.
Our algorithm consists of multiple components including network training, out-of-distribution detection, clustering, merge and refine steps.
Our experiments demonstrate the effectiveness of our approach to discover new GANs and can be used in an open-world setup.
- Score: 18.10496076534083
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: With the recent progress in Generative Adversarial Networks (GANs), it is
imperative for media and visual forensics to develop detectors which can
identify and attribute images to the model generating them. Existing works have
shown to attribute images to their corresponding GAN sources with high
accuracy. However, these works are limited to a closed set scenario, failing to
generalize to GANs unseen during train time and are therefore, not scalable
with a steady influx of new GANs. We present an iterative algorithm for
discovering images generated from previously unseen GANs by exploiting the fact
that all GANs leave distinct fingerprints on their generated images. Our
algorithm consists of multiple components including network training,
out-of-distribution detection, clustering, merge and refine steps. Through
extensive experiments, we show that our algorithm discovers unseen GANs with
high accuracy and also generalizes to GANs trained on unseen real datasets. We
additionally apply our algorithm to attribution and discovery of GANs in an
online fashion as well as to the more standard task of real/fake detection. Our
experiments demonstrate the effectiveness of our approach to discover new GANs
and can be used in an open-world setup.
Related papers
- Contrasting Deepfakes Diffusion via Contrastive Learning and Global-Local Similarities [88.398085358514]
Contrastive Deepfake Embeddings (CoDE) is a novel embedding space specifically designed for deepfake detection.
CoDE is trained via contrastive learning by additionally enforcing global-local similarities.
arXiv Detail & Related papers (2024-07-29T18:00:10Z) - GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross Appearance-Edge Learning [50.7702397913573]
The rapid advancement of photorealistic generators has reached a critical juncture where the discrepancy between authentic and manipulated images is increasingly indistinguishable.
Although there have been a number of publicly available face forgery datasets, the forgery faces are mostly generated using GAN-based synthesis technology.
We propose a large-scale, diverse, and fine-grained high-fidelity dataset, namely GenFace, to facilitate the advancement of deepfake detection.
arXiv Detail & Related papers (2024-02-03T03:13:50Z) - Rethinking the Up-Sampling Operations in CNN-based Generative Network
for Generalizable Deepfake Detection [86.97062579515833]
We introduce the concept of Neighboring Pixel Relationships(NPR) as a means to capture and characterize the generalized structural artifacts stemming from up-sampling operations.
A comprehensive analysis is conducted on an open-world dataset, comprising samples generated by tft28 distinct generative models.
This analysis culminates in the establishment of a novel state-of-the-art performance, showcasing a remarkable tft11.6% improvement over existing methods.
arXiv Detail & Related papers (2023-12-16T14:27:06Z) - Spider GAN: Leveraging Friendly Neighbors to Accelerate GAN Training [20.03447539784024]
We propose a novel approach for training GANs with images as inputs, but without enforcing any pairwise constraints.
The process can be made efficient by identifying closely related datasets, or a friendly neighborhood'' of the target distribution.
We show that the Spider GAN formulation results in faster convergence, as the generator can discover correspondence even between seemingly unrelated datasets.
arXiv Detail & Related papers (2023-05-12T17:03:18Z) - Extracting Semantic Knowledge from GANs with Unsupervised Learning [65.32631025780631]
Generative Adversarial Networks (GANs) encode semantics in feature maps in a linearly separable form.
We propose a novel clustering algorithm, named KLiSH, which leverages the linear separability to cluster GAN's features.
KLiSH succeeds in extracting fine-grained semantics of GANs trained on datasets of various objects.
arXiv Detail & Related papers (2022-11-30T03:18:16Z) - Distilling Representations from GAN Generator via Squeeze and Span [55.76208869775715]
We propose to distill knowledge from GAN generators by squeezing and spanning their representations.
We span the distilled representation of the synthetic domain to the real domain by also using real training data to remedy the mode collapse of GANs.
arXiv Detail & Related papers (2022-11-06T01:10:28Z) - GAN-Supervised Dense Visual Alignment [95.37027391102684]
We propose GAN-Supervised Learning, a framework for learning discriminative models and their GAN-generated training data jointly end-to-end.
Inspired by the classic Congealing method, our GANgealing algorithm trains a Spatial Transformer to map random samples from a GAN trained on unaligned data to a common, jointly-learned target mode.
arXiv Detail & Related papers (2021-12-09T18:59:58Z) - T-GD: Transferable GAN-generated Images Detection Framework [16.725880610265378]
We present the Transferable GAN-images Detection framework T-GD.
T-GD is composed of a teacher and a student model that can iteratively teach and evaluate each other to improve the detection performance.
To train the student model, we inject noise by mixing up the source and target datasets, while constraining the weight variation to preserve the starting point.
arXiv Detail & Related papers (2020-08-10T13:20:19Z) - Detection, Attribution and Localization of GAN Generated Images [24.430919035100317]
We propose a novel approach to detect, attribute and localize GAN generated images.
A deep learning network is then trained on these features to detect, attribute and localize these GAN generated/manipulated images.
A large scale evaluation of our approach on 5 GAN datasets shows promising results in detecting GAN generated images.
arXiv Detail & Related papers (2020-07-20T20:49:34Z) - DeepFake Detection by Analyzing Convolutional Traces [0.0]
We focus on the analysis of Deepfakes of human faces with the objective of creating a new detection method.
The proposed technique, by means of an Expectation Maximization (EM) algorithm, extracts a set of local features specifically addressed to model the underlying convolutional generative process.
Results demonstrated the effectiveness of the technique in distinguishing the different architectures and the corresponding generation process.
arXiv Detail & Related papers (2020-04-22T09:02:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.