Deepfake for the Good: Generating Avatars through Face-Swapping with Implicit Deepfake Generation

• URL: http://arxiv.org/abs/2402.06390v2
• Date: Mon, 02 Dec 2024 15:04:25 GMT
• Title: Deepfake for the Good: Generating Avatars through Face-Swapping with Implicit Deepfake Generation
• Authors: Georgii Stanishevskii, Jakub Steczkiewicz, Tomasz Szczepanik, Sławomir Tadeja, Jacek Tabor, Przemysław Spurek,
• Abstract summary: We show how to combine emerging deep-learning techniques to produce plausible 3D deepfake-based avatars. Our ImplicitDeepfake uses the classical deepfake algorithm to modify all training images separately and then train NeRF and GS on modified faces.
• Score: 6.720605329045581
• License:
• Abstract: Numerous emerging deep-learning techniques have had a substantial impact on computer graphics. Among the most promising breakthroughs are the rise of Neural Radiance Fields (NeRFs) and Gaussian Splatting (GS). NeRFs encode the object's shape and color in neural network weights using a handful of images with known camera positions to generate novel views. In contrast, GS provides accelerated training and inference without a decrease in rendering quality by encoding the object's characteristics in a collection of Gaussian distributions. These two techniques have found many use cases in spatial computing and other domains. On the other hand, the emergence of deepfake methods has sparked considerable controversy. Deepfakes refers to artificial intelligence-generated videos that closely mimic authentic footage. Using generative models, they can modify facial features, enabling the creation of altered identities or expressions that exhibit a remarkably realistic appearance to a real person. Despite these controversies, deepfake can offer a next-generation solution for avatar creation and gaming when of desirable quality. To that end, we show how to combine all these emerging technologies to obtain a more plausible outcome. Our ImplicitDeepfake uses the classical deepfake algorithm to modify all training images separately and then train NeRF and GS on modified faces. Such simple strategies can produce plausible 3D deepfake-based avatars.

Related papers

• Generalizable and Animatable Gaussian Head Avatar [50.34788590904843]
  We propose Generalizable and Animatable Gaussian head Avatar (GAGAvatar) for one-shot animatable head avatar reconstruction. We generate the parameters of 3D Gaussians from a single image in a single forward pass. Our method exhibits superior performance compared to previous methods in terms of reconstruction quality and expression accuracy.
  arXiv  Detail & Related papers  (2024-10-10T14:29:00Z)
• Deepfake detection in videos with multiple faces using geometric-fakeness features [79.16635054977068]
  Deepfakes of victims or public figures can be used by fraudsters for blackmailing, extorsion and financial fraud. In our research we propose to use geometric-fakeness features (GFF) that characterize a dynamic degree of a face presence in a video. We employ our approach to analyze videos with multiple faces that are simultaneously present in a video.
  arXiv  Detail & Related papers  (2024-10-10T13:10:34Z)
• Solutions to Deepfakes: Can Camera Hardware, Cryptography, and Deep Learning Verify Real Images? [51.3344199560726]
  It is imperative to establish methods that can separate real data from synthetic data with high confidence. This document aims to: present known strategies in detection and cryptography that can be employed to verify which images are real.
  arXiv  Detail & Related papers  (2024-07-04T22:01:21Z)
• Closing the Visual Sim-to-Real Gap with Object-Composable NeRFs [59.12526668734703]
  We introduce Composable Object Volume NeRF (COV-NeRF), an object-composable NeRF model that is the centerpiece of a real-to-sim pipeline. COV-NeRF extracts objects from real images and composes them into new scenes, generating photorealistic renderings and many types of 2D and 3D supervision.
  arXiv  Detail & Related papers  (2024-03-07T00:00:02Z)
• Text-image guided Diffusion Model for generating Deepfake celebrity interactions [50.37578424163951]
  Diffusion models have recently demonstrated highly realistic visual content generation. This paper devises and explores a novel method in that regard. Our results show that with the devised scheme, it is possible to create fake visual content with alarming realism.
  arXiv  Detail & Related papers  (2023-09-26T08:24:37Z)
• Copy Motion From One to Another: Fake Motion Video Generation [53.676020148034034]
  A compelling application of artificial intelligence is to generate a video of a target person performing arbitrary desired motion. Current methods typically employ GANs with a L2 loss to assess the authenticity of the generated videos. We propose a theoretically motivated Gromov-Wasserstein loss that facilitates learning the mapping from a pose to a foreground image. Our method is able to generate realistic target person videos, faithfully copying complex motions from a source person.
  arXiv  Detail & Related papers  (2022-05-03T08:45:22Z)
• DA-FDFtNet: Dual Attention Fake Detection Fine-tuning Network to Detect Various AI-Generated Fake Images [21.030153777110026]
  It has been much easier to create fake images such as "Deepfakes" Recent research has introduced few-shot learning, which uses a small amount of training data to produce fake images and videos more effectively. In this work, we propose Dual Attention Fine-tuning Network (DA-tNet) to detect the manipulated fake face images.
  arXiv  Detail & Related papers  (2021-12-22T16:25:24Z)
• Detecting Deepfake Videos Using Euler Video Magnification [1.8506048493564673]
  Deepfake videos are manipulating videos using advanced machine learning techniques. In this paper, we examine a technique for possible identification of deepfake videos. Our approach uses features extracted from the Euler technique to train three models to classify counterfeit and unaltered videos.
  arXiv  Detail & Related papers  (2021-01-27T17:37:23Z)
• Using GANs to Synthesise Minimum Training Data for Deepfake Generation [2.7393821783237184]
  Deepfakes can be useful in applications like entertainment, customer relations, or even assistive care. One problem with generating deepfakes is the requirement for a lot of image training data of the subject. We exploit the property of a GAN to produce images of an individual with variable facial expressions which we then use to generate a deepfake.
  arXiv  Detail & Related papers  (2020-11-10T22:05:38Z)
• Fighting Deepfake by Exposing the Convolutional Traces on Images [0.0]
  Mobile apps like FACEAPP make use of the most advanced Generative Adversarial Networks (GAN) to produce extreme transformations on human face photos. This kind of media object took the name of Deepfake and raised a new challenge in the multimedia forensics field: the Deepfake detection challenge. In this paper, a new approach aimed to extract a Deepfake fingerprint from images is proposed.
  arXiv  Detail & Related papers  (2020-08-07T08:49:23Z)
• Defending against GAN-based Deepfake Attacks via Transformation-aware Adversarial Faces [36.87244915810356]
  Deepfake represents a category of face-swapping attacks that leverage machine learning models. We propose to use novel transformation-aware adversarially perturbed faces as a defense against Deepfake attacks. We also propose to use an ensemble-based approach to enhance the defense robustness against GAN-based Deepfake variants.
  arXiv  Detail & Related papers  (2020-06-12T18:51:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.