Anisotropic multiresolution analyses for deep fake detection

• URL: http://arxiv.org/abs/2210.14874v1
• Date: Wed, 26 Oct 2022 17:26:09 GMT
• Title: Anisotropic multiresolution analyses for deep fake detection
• Authors: Wei Huang and Michelangelo Valsecchi and Michael Multerer
• Abstract summary: Generative Adversarial Networks (GANs) have paved the path towards entirely new media generation capabilities. They can also be misused and abused to fabricate elaborate lies, capable of stirring up the public debate. Previous studies have tackled this task by using classical machine learning techniques, such as k-nearest neighbours and eigenfaces. We argue that, since GANs primarily utilize isotropic convolutions to generate their output, they leave clear traces, their fingerprint, in the coefficient distribution on sub-bands extracted by anisotropic transformations.
• Score: 4.903718320156974
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative Adversarial Networks (GANs) have paved the path towards entirely new media generation capabilities at the forefront of image, video, and audio synthesis. However, they can also be misused and abused to fabricate elaborate lies, capable of stirring up the public debate. The threat posed by GANs has sparked the need to discern between genuine content and fabricated one. Previous studies have tackled this task by using classical machine learning techniques, such as k-nearest neighbours and eigenfaces, which unfortunately did not prove very effective. Subsequent methods have focused on leveraging on frequency decompositions, i.e., discrete cosine transform, wavelets, and wavelet packets, to preprocess the input features for classifiers. However, existing approaches only rely on isotropic transformations. We argue that, since GANs primarily utilize isotropic convolutions to generate their output, they leave clear traces, their fingerprint, in the coefficient distribution on sub-bands extracted by anisotropic transformations. We employ the fully separable wavelet transform and multiwavelets to obtain the anisotropic features to feed to standard CNN classifiers. Lastly, we find the fully separable transform capable of improving the state-of-the-art.

Related papers

• Tilt your Head: Activating the Hidden Spatial-Invariance of Classifiers [0.7704032792820767]
  Deep neural networks are applied in more and more areas of everyday life. They still lack essential abilities, such as robustly dealing with spatially transformed input signals. We propose a novel technique to emulate such an inference process for neural nets.
  arXiv  Detail & Related papers  (2024-05-06T09:47:29Z)
• Short-Time Fourier Transform for deblurring Variational Autoencoders [0.0]
  Variational Autoencoders (VAEs) are powerful generative models. Their generated samples are known to suffer from a characteristic blurriness, as compared to the outputs of alternative generating techniques.
  arXiv  Detail & Related papers  (2024-01-06T08:57:11Z)
• Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection [86.97062579515833]
  We introduce the concept of Neighboring Pixel Relationships(NPR) as a means to capture and characterize the generalized structural artifacts stemming from up-sampling operations. A comprehensive analysis is conducted on an open-world dataset, comprising samples generated by tft28 distinct generative models. This analysis culminates in the establishment of a novel state-of-the-art performance, showcasing a remarkable tft11.6% improvement over existing methods.
  arXiv  Detail & Related papers  (2023-12-16T14:27:06Z)
• Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation [105.22961467028234]
  Skip connections and normalisation layers are ubiquitous for the training of Deep Neural Networks (DNNs) Recent approaches such as Deep Kernel Shaping have made progress towards reducing our reliance on them. But these approaches are incompatible with the self-attention layers present in transformers.
  arXiv  Detail & Related papers  (2023-02-20T21:26:25Z)
• Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction [138.04956118993934]
  We propose a novel Transformer-based method, coarse-to-fine sparse Transformer (CST) CST embedding HSI sparsity into deep learning for HSI reconstruction. In particular, CST uses our proposed spectra-aware screening mechanism (SASM) for coarse patch selecting. Then the selected patches are fed into our customized spectra-aggregation hashing multi-head self-attention (SAH-MSA) for fine pixel clustering and self-similarity capturing.
  arXiv  Detail & Related papers  (2022-03-09T16:17:47Z)
• GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation [102.85440102147267]
  We propose a novel generative model named GeoDiff for molecular conformation prediction. We show that GeoDiff is superior or comparable to existing state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-03-06T09:47:01Z)
• Simpler is better: spectral regularization and up-sampling techniques for variational autoencoders [1.2234742322758418]
  characterization of the spectral behavior of generative models based on neural networks remains an open issue. Recent research has focused heavily on generative adversarial networks and the high-frequency discrepancies between real and generated images. We propose a simple 2D Fourier transform-based spectral regularization loss for the Variational Autoencoders (VAEs)
  arXiv  Detail & Related papers  (2022-01-19T11:49:57Z)
• Ensembling with Deep Generative Views [72.70801582346344]
  generative models can synthesize "views" of artificial images that mimic real-world variations, such as changes in color or pose. Here, we investigate whether such views can be applied to real images to benefit downstream analysis tasks such as image classification. We use StyleGAN2 as the source of generative augmentations and investigate this setup on classification tasks involving facial attributes, cat faces, and cars.
  arXiv  Detail & Related papers  (2021-04-29T17:58:35Z)
• GANs with Variational Entropy Regularizers: Applications in Mitigating the Mode-Collapse Issue [95.23775347605923]
  Building on the success of deep learning, Generative Adversarial Networks (GANs) provide a modern approach to learn a probability distribution from observed samples. GANs often suffer from the mode collapse issue where the generator fails to capture all existing modes of the input distribution. We take an information-theoretic approach and maximize a variational lower bound on the entropy of the generated samples to increase their diversity.
  arXiv  Detail & Related papers  (2020-09-24T19:34:37Z)
• Non-parallel Emotion Conversion using a Deep-Generative Hybrid Network and an Adversarial Pair Discriminator [16.18921154013272]
  We introduce a novel method for emotion conversion in speech that does not require parallel training data. Unlike the conventional cycle-GAN, our discriminator classifies whether a pair of input real and generated samples corresponds to the desired emotion conversion. We show that our model generalizes to new speakers by modifying speech produced by Wavenet.
  arXiv  Detail & Related papers  (2020-07-25T13:50:00Z)
• Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributions [13.439086686599891]
  We show that up-convolution or transposed convolution, are causing the inability of such models to reproduce spectral distributions of natural training data correctly. We propose to add a novel spectral regularization term to the training optimization objective. We show that this approach not only allows to train spectral consistent GANs that are avoiding high frequency errors.
  arXiv  Detail & Related papers  (2020-03-03T23:04:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.