Test-Time Augmentation for Pose-invariant Face Recognition

• URL: http://arxiv.org/abs/2505.09256v1
• Date: Wed, 14 May 2025 10:11:35 GMT
• Title: Test-Time Augmentation for Pose-invariant Face Recognition
• Authors: Jaemin Jung, Youngjoon Jang, Joon Son Chung,
• Abstract summary: Pose-TTA is a novel approach that aligns faces at inference time without additional training.<n>To achieve this, we employ a portrait animator that transfers the source image identity into the pose of a driving image.<n>We propose a weighted feature aggregation strategy to address any distortions or biases arising from the synthetic data.
• Score: 14.515296731166721
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The goal of this paper is to enhance face recognition performance by augmenting head poses during the testing phase. Existing methods often rely on training on frontalised images or learning pose-invariant representations, yet both approaches typically require re-training and testing for each dataset, involving a substantial amount of effort. In contrast, this study proposes Pose-TTA, a novel approach that aligns faces at inference time without additional training. To achieve this, we employ a portrait animator that transfers the source image identity into the pose of a driving image. Instead of frontalising a side-profile face -- which can introduce distortion -- Pose-TTA generates matching side-profile images for comparison, thereby reducing identity information loss. Furthermore, we propose a weighted feature aggregation strategy to address any distortions or biases arising from the synthetic data, thus enhancing the reliability of the augmented images. Extensive experiments on diverse datasets and with various pre-trained face recognition models demonstrate that Pose-TTA consistently improves inference performance. Moreover, our method is straightforward to integrate into existing face recognition pipelines, as it requires no retraining or fine-tuning of the underlying recognition models.

Related papers

• Pose-invariant face recognition via feature-space pose frontalization [9.105950041800225]
  Pose-invariant face recognition is a challenging problem for modern AI-based face recognition systems.<n>In this paper, a new method is presented to perform face frontalization and recognition within the feature space.<n>New training paradigm is proposed to maximize the potential of FSPFM and boost its performance.
  arXiv  Detail & Related papers  (2025-05-22T09:01:01Z)
• Understanding and Improving Training-Free AI-Generated Image Detections with Vision Foundation Models [68.90917438865078]
  Deepfake techniques for facial synthesis and editing pose serious risks for generative models.<n>In this paper, we investigate how detection performance varies across model backbones, types, and datasets.<n>We introduce Contrastive Blur, which enhances performance on facial images, and MINDER, which addresses noise type bias, balancing performance across domains.
  arXiv  Detail & Related papers  (2024-11-28T13:04:45Z)
• Realistic and Efficient Face Swapping: A Unified Approach with Diffusion Models [69.50286698375386]
  We propose a novel approach that better harnesses diffusion models for face-swapping. We introduce a mask shuffling technique during inpainting training, which allows us to create a so-called universal model for swapping. Ours is a relatively unified approach and so it is resilient to errors in other off-the-shelf models.
  arXiv  Detail & Related papers  (2024-09-11T13:43:53Z)
• Fiducial Focus Augmentation for Facial Landmark Detection [4.433764381081446]
  We propose a novel image augmentation technique to enhance the model's understanding of facial structures. We employ a Siamese architecture-based training mechanism with a Deep Canonical Correlation Analysis (DCCA)-based loss. Our approach outperforms multiple state-of-the-art approaches across various benchmark datasets.
  arXiv  Detail & Related papers  (2024-02-23T01:34:00Z)
• Effective Adapter for Face Recognition in the Wild [72.75516495170199]
  We tackle the challenge of face recognition in the wild, where images often suffer from low quality and real-world distortions. Traditional approaches-either training models directly on degraded images or their enhanced counterparts using face restoration techniques-have proven ineffective. We propose an effective adapter for augmenting existing face recognition models trained on high-quality facial datasets.
  arXiv  Detail & Related papers  (2023-12-04T08:55:46Z)
• X-Transfer: A Transfer Learning-Based Framework for GAN-Generated Fake Image Detection [33.31312811230408]
  misuse of GANs for generating deceptive images, such as face replacement, raises significant security concerns. This paper introduces a novel GAN-generated image detection algorithm called X-Transfer. It enhances transfer learning by utilizing two neural networks that employ interleaved parallel gradient transmission.
  arXiv  Detail & Related papers  (2023-10-07T01:23:49Z)
• FACE-AUDITOR: Data Auditing in Facial Recognition Systems [24.082527732931677]
  Few-shot-based facial recognition systems have gained increasing attention due to their scalability and ability to work with a few face images. To prevent the face images from being misused, one straightforward approach is to modify the raw face images before sharing them. We propose a complete toolkit FACE-AUDITOR that can query the few-shot-based facial recognition model and determine whether any of a user's face images is used in training the model.
  arXiv  Detail & Related papers  (2023-04-05T23:03:54Z)
• Pose-disentangled Contrastive Learning for Self-supervised Facial Representation [12.677909048435408]
  We propose a novel Pose-disentangled Contrastive Learning (PCL) method for general self-supervised facial representation. Our PCL first devises a pose-disentangled decoder (PDD), which disentangles the pose-related features from the face-aware features. We then introduce a pose-related contrastive learning scheme that learns pose-related information based on data augmentation of the same image.
  arXiv  Detail & Related papers  (2022-11-24T09:30:51Z)
• PoseFace: Pose-Invariant Features and Pose-Adaptive Loss for Face Recognition [42.62320574369969]
  We propose an efficient PoseFace framework which utilizes the facial landmarks to disentangle the pose-invariant features and exploits a pose-adaptive loss to handle the imbalance issue adaptively.
  arXiv  Detail & Related papers  (2021-07-25T03:50:47Z)
• Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection [65.92058628082322]
  Non-parametric face modeling aims to reconstruct 3D face only from images without shape assumptions. This paper presents a novel Learning to Aggregate and Personalize framework for unsupervised robust 3D face modeling.
  arXiv  Detail & Related papers  (2021-06-15T03:10:17Z)
• Cross-Resolution Adversarial Dual Network for Person Re-Identification and Beyond [59.149653740463435]
  Person re-identification (re-ID) aims at matching images of the same person across camera views. Due to varying distances between cameras and persons of interest, resolution mismatch can be expected. We propose a novel generative adversarial network to address cross-resolution person re-ID.
  arXiv  Detail & Related papers  (2020-02-19T07:21:38Z)
• Joint Deep Learning of Facial Expression Synthesis and Recognition [97.19528464266824]
  We propose a novel joint deep learning of facial expression synthesis and recognition method for effective FER. The proposed method involves a two-stage learning procedure. Firstly, a facial expression synthesis generative adversarial network (FESGAN) is pre-trained to generate facial images with different facial expressions. In order to alleviate the problem of data bias between the real images and the synthetic images, we propose an intra-class loss with a novel real data-guided back-propagation (RDBP) algorithm.
  arXiv  Detail & Related papers  (2020-02-06T10:56:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.