A recurrent cycle consistency loss for progressive face-to-face
synthesis
- URL: http://arxiv.org/abs/2004.07165v1
- Date: Tue, 14 Apr 2020 16:53:41 GMT
- Title: A recurrent cycle consistency loss for progressive face-to-face
synthesis
- Authors: Enrique Sanchez, Michel Valstar
- Abstract summary: This paper addresses a major flaw of the cycle consistency loss when used to preserve the input appearance in the face-to-face synthesis domain.
We show that the images generated by a network trained using this loss conceal a noise that hinders their use for further tasks.
We propose a ''recurrent cycle consistency loss'' which for different sequences of target attributes minimises the distance between the output images.
- Score: 5.71097144710995
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This paper addresses a major flaw of the cycle consistency loss when used to
preserve the input appearance in the face-to-face synthesis domain. In
particular, we show that the images generated by a network trained using this
loss conceal a noise that hinders their use for further tasks. To overcome this
limitation, we propose a ''recurrent cycle consistency loss" which for
different sequences of target attributes minimises the distance between the
output images, independent of any intermediate step. We empirically validate
not only that our loss enables the re-use of generated images, but that it also
improves their quality. In addition, we propose the very first network that
covers the task of unconstrained landmark-guided face-to-face synthesis.
Contrary to previous works, our proposed approach enables the transfer of a
particular set of input features to a large span of poses and expressions,
whereby the target landmarks become the ground-truth points. We then evaluate
the consistency of our proposed approach to synthesise faces at the target
landmarks. To the best of our knowledge, we are the first to propose a loss to
overcome the limitation of the cycle consistency loss, and the first to propose
an ''in-the-wild'' landmark guided synthesis approach. Code and models for this
paper can be found in https://github.com/ESanchezLozano/GANnotation
Related papers
- Fine-Grained Alignment and Noise Refinement for Compositional Text-to-Image Generation [2.1457109220047137]
We present an innovative, training-free method that incorporates tailored objectives to account for textual constraints.
Our method, relying solely on our proposed objective functions, significantly enhances compositionality, achieving a 24% improvement in human evaluation.
Our fine-grained noise refinement proves effective, boosting performance by up to 5%.
arXiv Detail & Related papers (2025-03-09T08:18:43Z) - Consistent Human Image and Video Generation with Spatially Conditioned Diffusion [82.4097906779699]
Consistent human-centric image and video synthesis aims to generate images with new poses while preserving appearance consistency with a given reference image.
We frame the task as a spatially-conditioned inpainting problem, where the target image is in-painted to maintain appearance consistency with the reference.
This approach enables the reference features to guide the generation of pose-compliant targets within a unified denoising network.
arXiv Detail & Related papers (2024-12-19T05:02:30Z) - Occlusion Resilient 3D Human Pose Estimation [52.49366182230432]
Occlusions remain one of the key challenges in 3D body pose estimation from single-camera video sequences.
We demonstrate the effectiveness of this approach compared to state-of-the-art techniques that infer poses from single-camera sequences.
arXiv Detail & Related papers (2024-02-16T19:29:43Z) - Robustness-Guided Image Synthesis for Data-Free Quantization [15.91924736452861]
We propose Robustness-Guided Image Synthesis (RIS) to enrich the semantics of synthetic images and improve image diversity.
RIS is a simple but effective method to enrich the semantics of synthetic images and improve image diversity.
We achieve state-of-the-art performance for various settings on data-free quantization and can be extended to other data-free compression tasks.
arXiv Detail & Related papers (2023-10-05T16:39:14Z) - Attribute-preserving Face Dataset Anonymization via Latent Code
Optimization [64.4569739006591]
We present a task-agnostic anonymization procedure that directly optimize the images' latent representation in the latent space of a pre-trained GAN.
We demonstrate through a series of experiments that our method is capable of anonymizing the identity of the images whilst -- crucially -- better-preserving the facial attributes.
arXiv Detail & Related papers (2023-03-20T17:34:05Z) - SD-GAN: Semantic Decomposition for Face Image Synthesis with Discrete
Attribute [0.0]
We propose an innovative framework to tackle challenging facial discrete attribute synthesis via semantic decomposing, dubbed SD-GAN.
The fusion network integrates 3D embedding for better identity preservation and discrete attribute synthesis.
We construct a large and valuable dataset MEGN for completing the lack of discrete attributes in the existing dataset.
arXiv Detail & Related papers (2022-07-12T04:23:38Z) - Gait Cycle Reconstruction and Human Identification from Occluded
Sequences [2.198430261120653]
We propose an effective neural network-based model to reconstruct the occluded frames in an input sequence before carrying out gait recognition.
We employ LSTM networks to predict an embedding for each occluded frame both from the forward and the backward directions.
While the LSTMs are trained to minimize the mean-squared loss, the fusion network is trained to optimize the pixel-wise cross-entropy loss between the ground-truth and the reconstructed samples.
arXiv Detail & Related papers (2022-06-20T16:04:31Z) - Fine-grained Identity Preserving Landmark Synthesis for Face Reenactment [30.062379710262068]
A landmark synthesis network is designed to generate fine-grained landmark faces with more details.
The network refines the manipulated landmarks and generates a smooth and gradually changing face landmark sequence with good identity preserving ability.
Experiments are conducted on our self-collected BeautySelfie and the public VoxCeleb1 datasets.
arXiv Detail & Related papers (2021-10-10T05:25:23Z) - Learned Spatial Representations for Few-shot Talking-Head Synthesis [68.3787368024951]
We propose a novel approach for few-shot talking-head synthesis.
We show that this disentangled representation leads to a significant improvement over previous methods.
arXiv Detail & Related papers (2021-04-29T17:59:42Z) - You Only Need Adversarial Supervision for Semantic Image Synthesis [84.83711654797342]
We propose a novel, simplified GAN model, which needs only adversarial supervision to achieve high quality results.
We show that images synthesized by our model are more diverse and follow the color and texture of real images more closely.
arXiv Detail & Related papers (2020-12-08T23:00:48Z) - Image Fine-grained Inpainting [89.17316318927621]
We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields.
To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss.
We also employ a discriminator with local and global branches to ensure local-global contents consistency.
arXiv Detail & Related papers (2020-02-07T03:45:25Z) - Exploiting Semantics for Face Image Deblurring [121.44928934662063]
We propose an effective and efficient face deblurring algorithm by exploiting semantic cues via deep convolutional neural networks.
We incorporate face semantic labels as input priors and propose an adaptive structural loss to regularize facial local structures.
The proposed method restores sharp images with more accurate facial features and details.
arXiv Detail & Related papers (2020-01-19T13:06:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.