Pixel-based Facial Expression Synthesis

• URL: http://arxiv.org/abs/2010.14397v1
• Date: Tue, 27 Oct 2020 16:00:45 GMT
• Title: Pixel-based Facial Expression Synthesis
• Authors: Arbish Akram, Nazar Khan
• Abstract summary: We propose a pixel-based facial expression synthesis method in which each output pixel observes only one input pixel. The proposed model is two orders of magnitude smaller which makes it suitable for deployment on resource-constrained devices.
• Score: 1.7056768055368383
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Facial expression synthesis has achieved remarkable advances with the advent of Generative Adversarial Networks (GANs). However, GAN-based approaches mostly generate photo-realistic results as long as the testing data distribution is close to the training data distribution. The quality of GAN results significantly degrades when testing images are from a slightly different distribution. Moreover, recent work has shown that facial expressions can be synthesized by changing localized face regions. In this work, we propose a pixel-based facial expression synthesis method in which each output pixel observes only one input pixel. The proposed method achieves good generalization capability by leveraging only a few hundred training images. Experimental results demonstrate that the proposed method performs comparably well against state-of-the-art GANs on in-dataset images and significantly better on out-of-dataset images. In addition, the proposed model is two orders of magnitude smaller which makes it suitable for deployment on resource-constrained devices.

Related papers

• Detection of Synthetic Face Images: Accuracy, Robustness, Generalization [1.757194730633422]
  We find that a simple model trained on a specific image generator can achieve near-perfect accuracy in separating synthetic and real images. The model turned out to be vulnerable to adversarial attacks and does not generalize to unseen generators.
  arXiv  Detail & Related papers  (2024-06-25T13:34:50Z)
• Rethinking Score Distillation as a Bridge Between Image Distributions [97.27476302077545]
  We show that our method seeks to transport corrupted images (source) to the natural image distribution (target) Our method can be easily applied across many domains, matching or beating the performance of specialized methods. We demonstrate its utility in text-to-2D, text-based NeRF optimization, translating paintings to real images, optical illusion generation, and 3D sketch-to-real.
  arXiv  Detail & Related papers  (2024-06-13T17:59:58Z)
• GAN-based Image Compression with Improved RDO Process [20.00340507091567]
  We present a novel GAN-based image compression approach with improved rate-distortion optimization process. To achieve this, we utilize the DISTS and MS-SSIM metrics to measure perceptual degeneration in color, texture, and structure. The proposed method outperforms the existing GAN-based methods and the state-of-the-art hybrid (i.e., VVC)
  arXiv  Detail & Related papers  (2023-06-18T03:21:11Z)
• Semantic Image Synthesis via Diffusion Models [159.4285444680301]
  Denoising Diffusion Probabilistic Models (DDPMs) have achieved remarkable success in various image generation tasks. Recent work on semantic image synthesis mainly follows the emphde facto Generative Adversarial Nets (GANs)
  arXiv  Detail & Related papers  (2022-06-30T18:31:51Z)
• Harmonizing Pathological and Normal Pixels for Pseudo-healthy Synthesis [68.5287824124996]
  We present a new type of discriminator, the segmentor, to accurately locate the lesions and improve the visual quality of pseudo-healthy images. We apply the generated images into medical image enhancement and utilize the enhanced results to cope with the low contrast problem. Comprehensive experiments on the T2 modality of BraTS demonstrate that the proposed method substantially outperforms the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-03-29T08:41:17Z)
• IMAGINE: Image Synthesis by Image-Guided Model Inversion [79.4691654458141]
  We introduce an inversion based method, denoted as IMAge-Guided model INvErsion (IMAGINE), to generate high-quality and diverse images. We leverage the knowledge of image semantics from a pre-trained classifier to achieve plausible generations. IMAGINE enables the synthesis procedure to simultaneously 1) enforce semantic specificity constraints during the synthesis, 2) produce realistic images without generator training, and 3) give users intuitive control over the generation process.
  arXiv  Detail & Related papers  (2021-04-13T02:00:24Z)
• Improving the Fairness of Deep Generative Models without Retraining [41.6580482370894]
  Generative Adversarial Networks (GANs) advance face synthesis through learning the underlying distribution of observed data. Despite the high-quality generated faces, some minority groups can be rarely generated from the trained models due to a biased image generation process. We propose an interpretable baseline method to balance the output facial attributes without retraining.
  arXiv  Detail & Related papers  (2020-12-09T03:20:41Z)
• Masked Linear Regression for Learning Local Receptive Fields for Facial Expression Synthesis [10.28711904929932]
  We propose a constrained version of ridge regression that exploits the local and sparse structure of facial expressions. In contrast to the existing approaches, our proposed model can be efficiently trained on larger image sizes. The proposed algorithm is also compared with state-of-the-art GANs including Pix2Pix, CycleGAN, StarGAN and GANimation.
  arXiv  Detail & Related papers  (2020-11-18T06:04:24Z)
• Pathological Retinal Region Segmentation From OCT Images Using Geometric Relation Based Augmentation [84.7571086566595]
  We propose improvements over previous GAN-based medical image synthesis methods by jointly encoding the intrinsic relationship of geometry and shape. The proposed method outperforms state-of-the-art segmentation methods on the public RETOUCH dataset having images captured from different acquisition procedures.
  arXiv  Detail & Related papers  (2020-03-31T11:50:43Z)
• Joint Deep Learning of Facial Expression Synthesis and Recognition [97.19528464266824]
  We propose a novel joint deep learning of facial expression synthesis and recognition method for effective FER. The proposed method involves a two-stage learning procedure. Firstly, a facial expression synthesis generative adversarial network (FESGAN) is pre-trained to generate facial images with different facial expressions. In order to alleviate the problem of data bias between the real images and the synthetic images, we propose an intra-class loss with a novel real data-guided back-propagation (RDBP) algorithm.
  arXiv  Detail & Related papers  (2020-02-06T10:56:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.