Self-supervised Auxiliary Learning for Texture and Model-based Hybrid Robust and Fair Featuring in Face Analysis

• URL: http://arxiv.org/abs/2409.19582v2
• Date: Tue, 5 Nov 2024 05:39:33 GMT
• Title: Self-supervised Auxiliary Learning for Texture and Model-based Hybrid Robust and Fair Featuring in Face Analysis
• Abstract summary: We explore Self-supervised Learning (SSL) as an auxiliary task to blend the texture-based local descriptors into feature modelling for efficient face analysis. We used the SSL task of mask auto-encoder (MAE) as an auxiliary task to reconstruct texture features such as local patterns along with the primary task for robust and unbiased face analysis.
• Score: 7.479920049311377
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we explore Self-supervised Learning (SSL) as an auxiliary task to blend the texture-based local descriptors into feature modelling for efficient face analysis. Combining a primary task and a self-supervised auxiliary task is beneficial for robust representation. Therefore, we used the SSL task of mask auto-encoder (MAE) as an auxiliary task to reconstruct texture features such as local patterns along with the primary task for robust and unbiased face analysis. We experimented with our hypothesis on three major paradigms of face analysis: face attribute and face-based emotion analysis, and deepfake detection. Our experiment results exhibit that better feature representation can be gleaned from our proposed model for fair and bias-less face analysis.

Related papers

• A Backbone Benchmarking Study on Self-supervised Learning as a Auxiliary Task with Texture-based Local Descriptors for Face Analysis [1.9269756235105742]
  We benchmark different backbones and study their impact for self-supervised learning (SSL) as an auxiliary task.<n>We employ different shallow to deep backbones for the SSL task of Masked Auto-Encoder (MAE) as an auxiliary objective.<n>For consistency of feature representation quality and generalisation capability across various face analysis paradigms, there is no unified backbone.
  arXiv  Detail & Related papers  (2026-03-23T16:49:50Z)
• Learning an Ensemble Token from Task-driven Priors in Facial Analysis [1.4228349888743608]
  We introduce ET-Fuser, a novel methodology for learning ensemble token.<n>We propose a robust prior unification learning method that generates a ensemble token within a self-attention mechanism.<n>Our results show improvements across a variety of facial analysis, with statistically significant enhancements observed in the feature representations.
  arXiv  Detail & Related papers  (2025-07-02T02:07:31Z)
• Towards Fair and Robust Face Parsing for Generative AI: A Multi-Objective Approach [10.00430939898858]
  We propose a multi-objective learning framework that optimize accuracy, fairness, and robustness in face parsing. Our results show that fairness-aware and robust segmentation improves photorealism and consistency in face generation. Our findings demonstrate that multi-objective face parsing improves demographic consistency and robustness, leading to higher-quality GAN-based synthesis.
  arXiv  Detail & Related papers  (2025-02-06T00:41:35Z)
• FSFM: A Generalizable Face Security Foundation Model via Self-Supervised Facial Representation Learning [27.34249750803211]
  We propose a self-supervised pretraining framework to learn fundamental representations of real face images. Our model transfers better than supervised pretraining, visual and facial self-supervised learning arts, and even outperforms task-specialized SOTA methods.
  arXiv  Detail & Related papers  (2024-12-16T17:58:45Z)
• UniForensics: Face Forgery Detection via General Facial Representation [60.5421627990707]
  High-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video network, with a meta-functional face classification for enriched facial representation.
  arXiv  Detail & Related papers  (2024-07-26T20:51:54Z)
• Task-adaptive Q-Face [75.15668556061772]
  We propose a novel task-adaptive multi-task face analysis method named as Q-Face. Q-Face simultaneously performs multiple face analysis tasks with a unified model. Our method achieves state-of-the-art performance on face expression recognition, action unit detection, face attribute analysis, age estimation, and face pose estimation.
  arXiv  Detail & Related papers  (2024-05-15T03:13:11Z)
• FaceXFormer: A Unified Transformer for Facial Analysis [59.94066615853198]
  FaceXformer is an end-to-end unified transformer model for a range of facial analysis tasks. Our model effectively handles images "in-the-wild," demonstrating its robustness and generalizability across eight different tasks.
  arXiv  Detail & Related papers  (2024-03-19T17:58:04Z)
• Faceptor: A Generalist Model for Face Perception [52.8066001012464]
  Faceptor is proposed to adopt a well-designed single-encoder dual-decoder architecture. Layer-Attention into Faceptor enables the model to adaptively select features from optimal layers to perform the desired tasks. Our training framework can also be applied to auxiliary supervised learning, significantly improving performance in data-sparse tasks such as age estimation and expression recognition.
  arXiv  Detail & Related papers  (2024-03-14T15:42:31Z)
• General Facial Representation Learning in a Visual-Linguistic Manner [45.92447707178299]
  We introduce a framework, called FaRL, for general Facial Representation Learning in a visual-linguistic manner. We show that FaRL achieves better transfer performance compared with previous pre-trained models. Our model surpasses the state-of-the-art methods on face analysis tasks including face parsing and face alignment.
  arXiv  Detail & Related papers  (2021-12-06T15:22:05Z)
• Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection [65.92058628082322]
  Non-parametric face modeling aims to reconstruct 3D face only from images without shape assumptions. This paper presents a novel Learning to Aggregate and Personalize framework for unsupervised robust 3D face modeling.
  arXiv  Detail & Related papers  (2021-06-15T03:10:17Z)
• MAFER: a Multi-resolution Approach to Facial Expression Recognition [9.878384185493623]
  We propose a two-step learning procedure, named MAFER, to train Deep Learning models tasked with recognizing facial expressions. A relevant feature of MAFER is that it is task-agnostic, i.e., it can be used complementarily to other objective-related techniques.
  arXiv  Detail & Related papers  (2021-05-06T07:26:58Z)
• InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs [73.27299786083424]
  We propose a framework called InterFaceGAN to interpret the disentangled face representation learned by state-of-the-art GAN models. We first find that GANs learn various semantics in some linear subspaces of the latent space. We then conduct a detailed study on the correlation between different semantics and manage to better disentangle them via subspace projection.
  arXiv  Detail & Related papers  (2020-05-18T18:01:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.