An Unsupervised Way to Understand Artifact Generating Internal Units in Generative Neural Networks

• URL: http://arxiv.org/abs/2112.08814v1
• Date: Thu, 16 Dec 2021 11:59:26 GMT
• Title: An Unsupervised Way to Understand Artifact Generating Internal Units in Generative Neural Networks
• Authors: Haedong Jeong, Jiyeon Han and Jaesik Choi
• Abstract summary: We propose the concept of local activation to detect artifact generations without additional supervision. We empirically verify that our approach can detect and correct artifact generations from GANs with various datasets.
• Score: 19.250873974729817
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Despite significant improvements on the image generation performance of Generative Adversarial Networks (GANs), generations with low visual fidelity still have been observed. As widely used metrics for GANs focus more on the overall performance of the model, evaluation on the quality of individual generations or detection of defective generations is challenging. While recent studies try to detect featuremap units that cause artifacts and evaluate individual samples, these approaches require additional resources such as external networks or a number of training data to approximate the real data manifold. In this work, we propose the concept of local activation, and devise a metric on the local activation to detect artifact generations without additional supervision. We empirically verify that our approach can detect and correct artifact generations from GANs with various datasets. Finally, we discuss a geometrical analysis to partially reveal the relation between the proposed concept and low visual fidelity.

Related papers

• Contrasting Deepfakes Diffusion via Contrastive Learning and Global-Local Similarities [88.398085358514]
  Contrastive Deepfake Embeddings (CoDE) is a novel embedding space specifically designed for deepfake detection. CoDE is trained via contrastive learning by additionally enforcing global-local similarities.
  arXiv  Detail & Related papers  (2024-07-29T18:00:10Z)
• GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross Appearance-Edge Learning [50.7702397913573]
  The rapid advancement of photorealistic generators has reached a critical juncture where the discrepancy between authentic and manipulated images is increasingly indistinguishable. Although there have been a number of publicly available face forgery datasets, the forgery faces are mostly generated using GAN-based synthesis technology. We propose a large-scale, diverse, and fine-grained high-fidelity dataset, namely GenFace, to facilitate the advancement of deepfake detection.
  arXiv  Detail & Related papers  (2024-02-03T03:13:50Z)
• Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection [86.97062579515833]
  We introduce the concept of Neighboring Pixel Relationships(NPR) as a means to capture and characterize the generalized structural artifacts stemming from up-sampling operations. A comprehensive analysis is conducted on an open-world dataset, comprising samples generated by tft28 distinct generative models. This analysis culminates in the establishment of a novel state-of-the-art performance, showcasing a remarkable tft11.6% improvement over existing methods.
  arXiv  Detail & Related papers  (2023-12-16T14:27:06Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• Entity-Conditioned Question Generation for Robust Attention Distribution in Neural Information Retrieval [51.53892300802014]
  We show that supervised neural information retrieval models are prone to learning sparse attention patterns over passage tokens. Using a novel targeted synthetic data generation method, we teach neural IR to attend more uniformly and robustly to all entities in a given passage.
  arXiv  Detail & Related papers  (2022-04-24T22:36:48Z)
• Generating unrepresented proportions of geological facies using Generative Adversarial Networks [0.0]
  We investigate the capacity of Generative Adversarial Networks (GANs) in interpolating and extrapolating facies proportions in a geological dataset. Specifically, we design a conditional GANs model that can drive the generated facies toward new proportions not found in the training set. The presented numerical experiments on images of binary and multiple facies showed good geological consistency as well as strong correlation with the target conditions.
  arXiv  Detail & Related papers  (2022-03-17T22:38:45Z)
• on the effectiveness of generative adversarial network on anomaly detection [1.6244541005112747]
  GANs rely on the rich contextual information of these models to identify the actual training distribution. We suggest a new unsupervised model based on GANs --a combination of an autoencoder and a GAN. A new scoring function was introduced to target anomalies where a linear combination of the internal representation of the discriminator and the generator's visual representation, plus the encoded representation of the autoencoder, come together to define the proposed anomaly score.
  arXiv  Detail & Related papers  (2021-12-31T16:35:47Z)
• Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning [50.24174211654775]
  We present a novel contrastive self-supervised learning framework for anomaly detection on attributed networks. Our framework fully exploits the local information from network data by sampling a novel type of contrastive instance pair. A graph neural network-based contrastive learning model is proposed to learn informative embedding from high-dimensional attributes and local structure.
  arXiv  Detail & Related papers  (2021-02-27T03:17:20Z)
• Joint Generative and Contrastive Learning for Unsupervised Person Re-identification [15.486689594217273]
  Recent self-supervised contrastive learning provides an effective approach for unsupervised person re-identification (ReID) In this paper, we incorporate a Generative Adversarial Network (GAN) and a contrastive learning module into one joint training framework.
  arXiv  Detail & Related papers  (2020-12-16T16:49:57Z)
• Generative Adversarial Networks (GANs): An Overview of Theoretical Model, Evaluation Metrics, and Recent Developments [9.023847175654602]
  Generative Adversarial Network (GAN) is an effective method to produce samples of large-scale data distribution. GANs provide an appropriate way to learn deep representations without widespread use of labeled training data. In GANs, the generative model is estimated via a competitive process where the generator and discriminator networks are trained simultaneously.
  arXiv  Detail & Related papers  (2020-05-27T05:56:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.