Learning Facial Representations from the Cycle-consistency of Face
- URL: http://arxiv.org/abs/2108.03427v1
- Date: Sat, 7 Aug 2021 11:30:35 GMT
- Title: Learning Facial Representations from the Cycle-consistency of Face
- Authors: Jia-Ren Chang, Yong-Sheng Chen, Wei-Chen Chiu
- Abstract summary: We introduce cycle-consistency in facial characteristics as free supervisory signal to learn facial representations from unlabeled facial images.
The learning is realized by superimposing the facial motion cycle-consistency and identity cycle-consistency constraints.
Our approach is competitive with those of existing methods, demonstrating the rich and unique information embedded in the disentangled representations.
- Score: 23.23272327438177
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Faces manifest large variations in many aspects, such as identity,
expression, pose, and face styling. Therefore, it is a great challenge to
disentangle and extract these characteristics from facial images, especially in
an unsupervised manner. In this work, we introduce cycle-consistency in facial
characteristics as free supervisory signal to learn facial representations from
unlabeled facial images. The learning is realized by superimposing the facial
motion cycle-consistency and identity cycle-consistency constraints. The main
idea of the facial motion cycle-consistency is that, given a face with
expression, we can perform de-expression to a neutral face via the removal of
facial motion and further perform re-expression to reconstruct back to the
original face. The main idea of the identity cycle-consistency is to exploit
both de-identity into mean face by depriving the given neutral face of its
identity via feature re-normalization and re-identity into neutral face by
adding the personal attributes to the mean face. At training time, our model
learns to disentangle two distinct facial representations to be useful for
performing cycle-consistent face reconstruction. At test time, we use the
linear protocol scheme for evaluating facial representations on various tasks,
including facial expression recognition and head pose regression. We also can
directly apply the learnt facial representations to person recognition,
frontalization and image-to-image translation. Our experiments show that the
results of our approach is competitive with those of existing methods,
demonstrating the rich and unique information embedded in the disentangled
representations. Code is available at https://github.com/JiaRenChang/FaceCycle .
Related papers
- ComFace: Facial Representation Learning with Synthetic Data for Comparing Faces [5.07975834105566]
We propose a facial representation learning method using synthetic images for comparing faces, called ComFace.
For effective representation learning, ComFace aims to acquire two feature representations, i.e., inter-personal facial differences and intra-personal facial changes.
Our ComFace, trained using only synthetic data, achieves comparable to or better transfer performance than general pre-training and state-of-the-art representation learning methods trained using real images.
arXiv Detail & Related papers (2024-05-25T02:44:07Z) - LAFS: Landmark-based Facial Self-supervised Learning for Face
Recognition [37.4550614524874]
We focus on learning facial representations that can be adapted to train effective face recognition models.
We explore the learning strategy of unlabeled facial images through self-supervised pretraining.
Our method achieves significant improvement over the state-of-the-art on multiple face recognition benchmarks.
arXiv Detail & Related papers (2024-03-13T01:07:55Z) - Self-Supervised Facial Representation Learning with Facial Region
Awareness [13.06996608324306]
Self-supervised pre-training has been proven to be effective in learning transferable representations that benefit various visual tasks.
Recent efforts toward this goal are limited to treating each face image as a whole.
We propose a novel self-supervised facial representation learning framework to learn consistent global and local facial representations.
arXiv Detail & Related papers (2024-03-04T15:48:56Z) - A Generalist FaceX via Learning Unified Facial Representation [77.74407008931486]
FaceX is a novel facial generalist model capable of handling diverse facial tasks simultaneously.
Our versatile FaceX achieves competitive performance compared to elaborate task-specific models on popular facial editing tasks.
arXiv Detail & Related papers (2023-12-31T17:41:48Z) - Emotion Separation and Recognition from a Facial Expression by
Generating the Poker Face with Vision Transformers [57.67586172996843]
We propose a novel FER model, called Poker Face Vision Transformer or PF-ViT, to separate and recognize the disturbance-agnostic emotion from a static facial image.
PF-ViT generates its corresponding poker face without the need for paired images.
arXiv Detail & Related papers (2022-07-22T13:39:06Z) - SynFace: Face Recognition with Synthetic Data [83.15838126703719]
We devise the SynFace with identity mixup (IM) and domain mixup (DM) to mitigate the performance gap.
We also perform a systematically empirical analysis on synthetic face images to provide some insights on how to effectively utilize synthetic data for face recognition.
arXiv Detail & Related papers (2021-08-18T03:41:54Z) - Learning Fair Face Representation With Progressive Cross Transformer [79.73754444296213]
We propose a progressive cross transformer (PCT) method for fair face recognition.
We show that PCT is capable of mitigating bias in face recognition while achieving state-of-the-art FR performance.
arXiv Detail & Related papers (2021-08-11T01:31:14Z) - I Only Have Eyes for You: The Impact of Masks On Convolutional-Based
Facial Expression Recognition [78.07239208222599]
We evaluate how the recently proposed FaceChannel adapts towards recognizing facial expressions from persons with masks.
We also perform specific feature-level visualization to demonstrate how the inherent capabilities of the FaceChannel to learn and combine facial features change when in a constrained social interaction scenario.
arXiv Detail & Related papers (2021-04-16T20:03:30Z) - Facial Expressions as a Vulnerability in Face Recognition [73.85525896663371]
This work explores facial expression bias as a security vulnerability of face recognition systems.
We present a comprehensive analysis of how facial expression bias impacts the performance of face recognition technologies.
arXiv Detail & Related papers (2020-11-17T18:12:41Z) - LandmarkGAN: Synthesizing Faces from Landmarks [43.53204737135101]
We describe a new method, namely LandmarkGAN, to synthesize faces based on facial landmarks as input.
Our method is able to transform a set of facial landmarks into new faces of different subjects, while retains the same facial expression and orientation.
arXiv Detail & Related papers (2020-10-31T13:27:21Z) - FaR-GAN for One-Shot Face Reenactment [20.894596219099164]
We present a one-shot face reenactment model, FaR-GAN, that takes only one face image of any given source identity and a target expression as input.
The proposed method makes no assumptions about the source identity, facial expression, head pose, or even image background.
arXiv Detail & Related papers (2020-05-13T16:15:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.