Learning Facial Representations from the Cycle-consistency of Face

• URL: http://arxiv.org/abs/2108.03427v1
• Date: Sat, 7 Aug 2021 11:30:35 GMT
• Title: Learning Facial Representations from the Cycle-consistency of Face
• Authors: Jia-Ren Chang, Yong-Sheng Chen, Wei-Chen Chiu
• Abstract summary: We introduce cycle-consistency in facial characteristics as free supervisory signal to learn facial representations from unlabeled facial images. The learning is realized by superimposing the facial motion cycle-consistency and identity cycle-consistency constraints. Our approach is competitive with those of existing methods, demonstrating the rich and unique information embedded in the disentangled representations.
• Score: 23.23272327438177
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Faces manifest large variations in many aspects, such as identity, expression, pose, and face styling. Therefore, it is a great challenge to disentangle and extract these characteristics from facial images, especially in an unsupervised manner. In this work, we introduce cycle-consistency in facial characteristics as free supervisory signal to learn facial representations from unlabeled facial images. The learning is realized by superimposing the facial motion cycle-consistency and identity cycle-consistency constraints. The main idea of the facial motion cycle-consistency is that, given a face with expression, we can perform de-expression to a neutral face via the removal of facial motion and further perform re-expression to reconstruct back to the original face. The main idea of the identity cycle-consistency is to exploit both de-identity into mean face by depriving the given neutral face of its identity via feature re-normalization and re-identity into neutral face by adding the personal attributes to the mean face. At training time, our model learns to disentangle two distinct facial representations to be useful for performing cycle-consistent face reconstruction. At test time, we use the linear protocol scheme for evaluating facial representations on various tasks, including facial expression recognition and head pose regression. We also can directly apply the learnt facial representations to person recognition, frontalization and image-to-image translation. Our experiments show that the results of our approach is competitive with those of existing methods, demonstrating the rich and unique information embedded in the disentangled representations. Code is available at https://github.com/JiaRenChang/FaceCycle .

Related papers

• Knowledge-Enhanced Facial Expression Recognition with Emotional-to-Neutral Transformation [66.53435569574135]
  Existing facial expression recognition methods typically fine-tune a pre-trained visual encoder using discrete labels. We observe that the rich knowledge in text embeddings, generated by vision-language models, is a promising alternative for learning discriminative facial expression representations. We propose a novel knowledge-enhanced FER method with an emotional-to-neutral transformation.
  arXiv  Detail & Related papers  (2024-09-13T07:28:57Z)
• Towards Localized Fine-Grained Control for Facial Expression Generation [54.82883891478555]
  Humans, particularly their faces, are central to content generation due to their ability to convey rich expressions and intent. Current generative models mostly generate flat neutral expressions and characterless smiles without authenticity. We propose the use of AUs (action units) for facial expression control in face generation.
  arXiv  Detail & Related papers  (2024-07-25T18:29:48Z)
• ComFace: Facial Representation Learning with Synthetic Data for Comparing Faces [5.07975834105566]
  We propose a facial representation learning method using synthetic images for comparing faces, called ComFace. For effective representation learning, ComFace aims to acquire two feature representations, i.e., inter-personal facial differences and intra-personal facial changes. Our ComFace, trained using only synthetic data, achieves comparable to or better transfer performance than general pre-training and state-of-the-art representation learning methods trained using real images.
  arXiv  Detail & Related papers  (2024-05-25T02:44:07Z)
• LAFS: Landmark-based Facial Self-supervised Learning for Face Recognition [37.4550614524874]
  We focus on learning facial representations that can be adapted to train effective face recognition models. We explore the learning strategy of unlabeled facial images through self-supervised pretraining. Our method achieves significant improvement over the state-of-the-art on multiple face recognition benchmarks.
  arXiv  Detail & Related papers  (2024-03-13T01:07:55Z)
• Self-Supervised Facial Representation Learning with Facial Region Awareness [13.06996608324306]
  Self-supervised pre-training has been proven to be effective in learning transferable representations that benefit various visual tasks. Recent efforts toward this goal are limited to treating each face image as a whole. We propose a novel self-supervised facial representation learning framework to learn consistent global and local facial representations.
  arXiv  Detail & Related papers  (2024-03-04T15:48:56Z)
• A Generalist FaceX via Learning Unified Facial Representation [77.74407008931486]
  FaceX is a novel facial generalist model capable of handling diverse facial tasks simultaneously. Our versatile FaceX achieves competitive performance compared to elaborate task-specific models on popular facial editing tasks.
  arXiv  Detail & Related papers  (2023-12-31T17:41:48Z)
• Emotion Separation and Recognition from a Facial Expression by Generating the Poker Face with Vision Transformers [57.1091606948826]
  We propose a novel FER model, named Poker Face Vision Transformer or PF-ViT, to address these challenges. PF-ViT aims to separate and recognize the disturbance-agnostic emotion from a static facial image via generating its corresponding poker face. PF-ViT utilizes vanilla Vision Transformers, and its components are pre-trained as Masked Autoencoders on a large facial expression dataset.
  arXiv  Detail & Related papers  (2022-07-22T13:39:06Z)
• SynFace: Face Recognition with Synthetic Data [83.15838126703719]
  We devise the SynFace with identity mixup (IM) and domain mixup (DM) to mitigate the performance gap. We also perform a systematically empirical analysis on synthetic face images to provide some insights on how to effectively utilize synthetic data for face recognition.
  arXiv  Detail & Related papers  (2021-08-18T03:41:54Z)
• Learning Fair Face Representation With Progressive Cross Transformer [79.73754444296213]
  We propose a progressive cross transformer (PCT) method for fair face recognition. We show that PCT is capable of mitigating bias in face recognition while achieving state-of-the-art FR performance.
  arXiv  Detail & Related papers  (2021-08-11T01:31:14Z)
• LandmarkGAN: Synthesizing Faces from Landmarks [43.53204737135101]
  We describe a new method, namely LandmarkGAN, to synthesize faces based on facial landmarks as input. Our method is able to transform a set of facial landmarks into new faces of different subjects, while retains the same facial expression and orientation.
  arXiv  Detail & Related papers  (2020-10-31T13:27:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.