Investigation of Accuracy and Bias in Face Recognition Trained with Synthetic Data

• URL: http://arxiv.org/abs/2507.20782v1
• Date: Mon, 28 Jul 2025 12:52:23 GMT
• Title: Investigation of Accuracy and Bias in Face Recognition Trained with Synthetic Data
• Authors: Pavel Korshunov, Ketan Kotwal, Christophe Ecabert, Vidit Vidit, Amir Mohammadi, Sebastien Marcel,
• Abstract summary: We evaluate the impact of synthetic data on bias and performance of face recognition systems.<n>By maintaining equal identity count across synthetic and real datasets, we ensure fair comparisons.<n>Our results demonstrate that although synthetic data still lags behind the real datasets in the generalization on IJB-B/C, demographically balanced synthetic datasets, especially those generated with SD35, show potential for bias mitigation.
• Score: 10.241047069730058
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Synthetic data has emerged as a promising alternative for training face recognition (FR) models, offering advantages in scalability, privacy compliance, and potential for bias mitigation. However, critical questions remain on whether both high accuracy and fairness can be achieved with synthetic data. In this work, we evaluate the impact of synthetic data on bias and performance of FR systems. We generate balanced face dataset, FairFaceGen, using two state of the art text-to-image generators, Flux.1-dev and Stable Diffusion v3.5 (SD35), and combine them with several identity augmentation methods, including Arc2Face and four IP-Adapters. By maintaining equal identity count across synthetic and real datasets, we ensure fair comparisons when evaluating FR performance on standard (LFW, AgeDB-30, etc.) and challenging IJB-B/C benchmarks and FR bias on Racial Faces in-the-Wild (RFW) dataset. Our results demonstrate that although synthetic data still lags behind the real datasets in the generalization on IJB-B/C, demographically balanced synthetic datasets, especially those generated with SD35, show potential for bias mitigation. We also observe that the number and quality of intra-class augmentations significantly affect FR accuracy and fairness. These findings provide practical guidelines for constructing fairer FR systems using synthetic data.

Related papers

• FairCauseSyn: Towards Causally Fair LLM-Augmented Synthetic Data Generation [4.392938909804638]
  Synthetic data generation creates data based on real-world data using generative models.<n>We develop the first LLM-augmented synthetic data generation method to enhance causal fairness using real-world health data.<n>When trained on causally fair predictors, synthetic data reduces bias on the sensitive attribute by 70% compared to real data.
  arXiv  Detail & Related papers  (2025-06-23T19:59:26Z)
• AugGen: Synthetic Augmentation Can Improve Discriminative Models [14.680260279598045]
  Synthetic data generation offers a promising alternative to external datasets or pre-trained models.<n>In this paper, we introduce AugGen, a self-contained synthetic augmentation technique.<n>Our findings demonstrate that carefully integrated synthetic data can both mitigate privacy constraints and substantially enhance discriminative performance in face recognition.
  arXiv  Detail & Related papers  (2025-03-14T16:10:21Z)
• Can Synthetic Data be Fair and Private? A Comparative Study of Synthetic Data Generation and Fairness Algorithms [2.144088660722956]
  We find that the DEbiasing CAusal Fairness (DECAF) algorithm achieves the best balance between privacy and fairness.<n>Applying pre-processing fairness algorithms to synthetic data improves fairness even more than when applied to real data.
  arXiv  Detail & Related papers  (2025-01-03T12:35:58Z)
• Second FRCSyn-onGoing: Winning Solutions and Post-Challenge Analysis to Improve Face Recognition with Synthetic Data [104.30479583607918]
  2nd FRCSyn-onGoing challenge is based on the 2nd Face Recognition Challenge in the Era of Synthetic Data (FRCSyn), originally launched at CVPR 2024.<n>We focus on exploring the use of synthetic data both individually and in combination with real data to solve current challenges in face recognition.
  arXiv  Detail & Related papers  (2024-12-02T11:12:01Z)
• The Impact of Balancing Real and Synthetic Data on Accuracy and Fairness in Face Recognition [10.849598219674132]
  We investigate the impact of demographically balanced authentic and synthetic data, both individually and in combination, on the accuracy and fairness of face recognition models. Our findings emphasize two main points: (i) the increased effectiveness of training data generated by diffusion-based models in enhancing accuracy, whether used alone or combined with subsets of authentic data, and (ii) the minimal impact of incorporating balanced data from pre-trained generative methods on fairness.
  arXiv  Detail & Related papers  (2024-09-04T16:50:48Z)
• SDFR: Synthetic Data for Face Recognition Competition [51.9134406629509]
  Large-scale face recognition datasets are collected by crawling the Internet and without individuals' consent, raising legal, ethical, and privacy concerns. Recently several works proposed generating synthetic face recognition datasets to mitigate concerns in web-crawled face recognition datasets. This paper presents the summary of the Synthetic Data for Face Recognition (SDFR) Competition held in conjunction with the 18th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2024) The SDFR competition was split into two tasks, allowing participants to train face recognition systems using new synthetic datasets and/or existing ones.
  arXiv  Detail & Related papers  (2024-04-06T10:30:31Z)
• Reliability in Semantic Segmentation: Can We Use Synthetic Data? [69.28268603137546]
  We show for the first time how synthetic data can be specifically generated to assess comprehensively the real-world reliability of semantic segmentation models. This synthetic data is employed to evaluate the robustness of pretrained segmenters. We demonstrate how our approach can be utilized to enhance the calibration and OOD detection capabilities of segmenters.
  arXiv  Detail & Related papers  (2023-12-14T18:56:07Z)
• Synthetic data, real errors: how (not) to publish and use synthetic data [86.65594304109567]
  We show how the generative process affects the downstream ML task. We introduce Deep Generative Ensemble (DGE) to approximate the posterior distribution over the generative process model parameters.
  arXiv  Detail & Related papers  (2023-05-16T07:30:29Z)
• CAFE: Learning to Condense Dataset by Aligning Features [72.99394941348757]
  We propose a novel scheme to Condense dataset by Aligning FEatures (CAFE) At the heart of our approach is an effective strategy to align features from the real and synthetic data across various scales. We validate the proposed CAFE across various datasets, and demonstrate that it generally outperforms the state of the art.
  arXiv  Detail & Related papers  (2022-03-03T05:58:49Z)
• DECAF: Generating Fair Synthetic Data Using Causally-Aware Generative Networks [71.6879432974126]
  We introduce DECAF: a GAN-based fair synthetic data generator for tabular data. We show that DECAF successfully removes undesired bias and is capable of generating high-quality synthetic data. We provide theoretical guarantees on the generator's convergence and the fairness of downstream models.
  arXiv  Detail & Related papers  (2021-10-25T12:39:56Z)
• On the use of automatically generated synthetic image datasets for benchmarking face recognition [2.0196229393131726]
  Recent advances in Generative Adversarial Networks (GANs) provide a pathway to replace real datasets by synthetic datasets. Recent advances in Generative Adversarial Networks (GANs) to synthesize realistic face images provide a pathway to replace real datasets by synthetic datasets. benchmarking results on the synthetic dataset are a good substitution, often providing error rates and system ranking similar to the benchmarking on the real dataset.
  arXiv  Detail & Related papers  (2021-06-08T09:54:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.