SteerFace: Debiasing Synthetic Face Generation via Adaptive Residue Perturbation

• URL: http://arxiv.org/abs/2605.30894v1
• Date: Fri, 29 May 2026 06:29:07 GMT
• Title: SteerFace: Debiasing Synthetic Face Generation via Adaptive Residue Perturbation
• Abstract summary: This paper identifies visual tendency as a previously underexplored limitation, whereby synthetic data exhibit an unrealistic prevalence of visual attributes.<n>Visual tendency can be attributed to the generator's conditioning on identity embeddings, through which co-occurring residual visual cues are unintentionally absorbed into learned identity semantics.<n>To discourage the generator from exploiting such visual cues, this paper proposes SteerFace, a simple and efficient training framework that perturbs identity embeddings by steering them toward random directions on the embedding hypersphere.
• Score: 24.834863849265034
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The shortage of legally compliant data for face recognition training has sparked growing interest in using synthetic data as an alternative. While recent diffusion-based methods enable the generation of photorealistic face images with strong identity adherence and data diversity, their downstream recognition performance still exhibits a significant synthetic-real gap. This paper identifies visual tendency as a previously underexplored limitation, whereby synthetic data exhibit an unrealistic prevalence of visual attributes and thus deviate from the real-data distribution. Visual tendency can be attributed to the generator's conditioning on identity embeddings, through which co-occurring residual visual cues are unintentionally absorbed into learned identity semantics. To discourage the generator from exploiting such visual cues, this paper proposes SteerFace, a simple and efficient training framework that perturbs identity embeddings by steering them toward random orthogonal directions on the embedding hypersphere. The perturbation serves as an identity-preserving regularizer that penalizes the generator's reliance on non-identity components, as supported by theoretical analysis. This paper further introduces an adaptive strategy that learns perturbation strengths with both sample-wise preference and favorable overall statistics. Extensive experiments show that SteerFace effectively mitigates visual tendency, outperforms prior methods in downstream face recognition, and generalizes well across different training datasets and generation pipelines.

Related papers

• Limitations of Synthetic Data Generation in Specialized Data-Scarce Domains [11.861117372710714]
  This work focuses on domains where images differ substantially from common image datasets and additional data are expensive to obtain.<n>We evaluate the downstream performance improvements yielded by two schools of generative sparse data extension.<n>Across five trauma classification tasks using subject-wise train--validation splits, no generative approach consistently outperforms a strong non-generative baseline.
  arXiv  Detail & Related papers  (2026-08-13T19:43:31Z)
• IdentiFace: Multi-Modal Iterative Diffusion Framework for Identifiable Suspect Face Generation in Crime Investigations [9.548761889081904]
  We propose a novel diffusion-based framework for identifiable suspect face generation, called IdentiFace.<n>We show that IdentiFace achieves superior performance over existing methods in terms of identity retrieval and shows strong potential for practical applications.
  arXiv  Detail & Related papers  (2026-05-01T09:04:17Z)
• IDperturb: Enhancing Variation in Synthetic Face Generation via Angular Perturbation [17.433921935288577]
  Synthetic data has emerged as a practical alternative to authentic face datasets for training face recognition (FR) systems.<n>Recent advances in identity-conditional diffusion models have enabled the generation of photorealistic and identity-consistent face images.<n>We propose IDPERTURB, a simple yet effective geometric-driven sampling strategy to enhance diversity in synthetic face generation.
  arXiv  Detail & Related papers  (2026-02-21T13:23:26Z)
• Beyond Inference Intervention: Identity-Decoupled Diffusion for Face Anonymization [55.29071072675132]
  Face anonymization aims to conceal identity information while preserving non-identity attributes.<n>We propose textbfIDsuperscript2Face, a training-centric anonymization framework.<n>We show that IDtextsuperscript2Face outperforms existing methods in visual quality, identity suppression, and utility preservation.
  arXiv  Detail & Related papers  (2025-10-28T09:28:12Z)
• UIFace: Unleashing Inherent Model Capabilities to Enhance Intra-Class Diversity in Synthetic Face Recognition [42.86969216015855]
  Face recognition (FR) stands as one of the most crucial applications in computer vision.<n>We propose a framework to enhance intra-class diversity for synthetic face recognition, shortened as UIFace.<n> Experiments show that our method significantly surpasses previous approaches with even less training data and half the size of synthetic dataset.
  arXiv  Detail & Related papers  (2025-02-27T06:22:18Z)
• Fairer Analysis and Demographically Balanced Face Generation for Fairer Face Verification [69.04239222633795]
  Face recognition and verification are two computer vision tasks whose performances have advanced with the introduction of deep representations.<n>Ethical, legal, and technical challenges due to the sensitive nature of face data and biases in real-world training datasets hinder their development.<n>We introduce a new controlled generation pipeline that improves fairness.
  arXiv  Detail & Related papers  (2024-12-04T14:30:19Z)
• ID$^3$: Identity-Preserving-yet-Diversified Diffusion Models for Synthetic Face Recognition [60.15830516741776]
  Synthetic face recognition (SFR) aims to generate datasets that mimic the distribution of real face data. We introduce a diffusion-fueled SFR model termed $textID3$. $textID3$ employs an ID-preserving loss to generate diverse yet identity-consistent facial appearances.
  arXiv  Detail & Related papers  (2024-09-26T06:46:40Z)
• The Impact of Balancing Real and Synthetic Data on Accuracy and Fairness in Face Recognition [10.849598219674132]
  We investigate the impact of demographically balanced authentic and synthetic data, both individually and in combination, on the accuracy and fairness of face recognition models. Our findings emphasize two main points: (i) the increased effectiveness of training data generated by diffusion-based models in enhancing accuracy, whether used alone or combined with subsets of authentic data, and (ii) the minimal impact of incorporating balanced data from pre-trained generative methods on fairness.
  arXiv  Detail & Related papers  (2024-09-04T16:50:48Z)
• Toward Fairer Face Recognition Datasets [69.04239222633795]
  Face recognition and verification are computer vision tasks whose performance has progressed with the introduction of deep representations. Ethical, legal, and technical challenges due to the sensitive character of face data and biases in real training datasets hinder their development. We promote fairness by introducing a demographic attributes balancing mechanism in generated training datasets.
  arXiv  Detail & Related papers  (2024-06-24T12:33:21Z)
• Bias and Diversity in Synthetic-based Face Recognition [12.408456748469426]
  We investigate how the diversity of synthetic face recognition datasets compares to authentic datasets. We look at the distribution of gender, ethnicity, age, and head position. With regard to bias, it can be seen that the synthetic-based models share a similar bias behavior with the authentic-based models.
  arXiv  Detail & Related papers  (2023-11-07T13:12:34Z)
• Segue: Side-information Guided Generative Unlearnable Examples for Facial Privacy Protection in Real World [64.4289385463226]
  We propose Segue: Side-information guided generative unlearnable examples. To improve transferability, we introduce side information such as true labels and pseudo labels. It can resist JPEG compression, adversarial training, and some standard data augmentations.
  arXiv  Detail & Related papers  (2023-10-24T06:22:37Z)
• Inductive Biased Estimation: Learning Generalizations for Identity Transfer [64.4487809928537]
  This paper proposes an Errors-in-Variables Adapter (EVA) model to induce learning of proper generalizations. To better match the source face with the target situation in terms of pose, expression, and background factors, we model the bias as a causal effect of the target situation on source identity.
  arXiv  Detail & Related papers  (2021-10-04T17:10:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.