The Impact of Balancing Real and Synthetic Data on Accuracy and Fairness in Face Recognition

• URL: http://arxiv.org/abs/2409.02867v1
• Date: Wed, 4 Sep 2024 16:50:48 GMT
• Title: The Impact of Balancing Real and Synthetic Data on Accuracy and Fairness in Face Recognition
• Authors: Andrea Atzori, Pietro Cosseddu, Gianni Fenu, Mirko Marras,
• Abstract summary: We investigate the impact of demographically balanced authentic and synthetic data, both individually and in combination, on the accuracy and fairness of face recognition models. Our findings emphasize two main points: (i) the increased effectiveness of training data generated by diffusion-based models in enhancing accuracy, whether used alone or combined with subsets of authentic data, and (ii) the minimal impact of incorporating balanced data from pre-trained generative methods on fairness.
• Score: 10.849598219674132
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Over the recent years, the advancements in deep face recognition have fueled an increasing demand for large and diverse datasets. Nevertheless, the authentic data acquired to create those datasets is typically sourced from the web, which, in many cases, can lead to significant privacy issues due to the lack of explicit user consent. Furthermore, obtaining a demographically balanced, large dataset is even more difficult because of the natural imbalance in the distribution of images from different demographic groups. In this paper, we investigate the impact of demographically balanced authentic and synthetic data, both individually and in combination, on the accuracy and fairness of face recognition models. Initially, several generative methods were used to balance the demographic representations of the corresponding synthetic datasets. Then a state-of-the-art face encoder was trained and evaluated using (combinations of) synthetic and authentic images. Our findings emphasized two main points: (i) the increased effectiveness of training data generated by diffusion-based models in enhancing accuracy, whether used alone or combined with subsets of authentic data, and (ii) the minimal impact of incorporating balanced data from pre-trained generative methods on fairness (in nearly all tested scenarios using combined datasets, fairness scores remained either unchanged or worsened, even when compared to unbalanced authentic datasets). Source code and data are available at \url{https://cutt.ly/AeQy1K5G} for reproducibility.

Related papers

• Toward Fairer Face Recognition Datasets [69.04239222633795]
  Face recognition and verification are computer vision tasks whose performance has progressed with the introduction of deep representations. Ethical, legal, and technical challenges due to the sensitive character of face data and biases in real training datasets hinder their development. We promote fairness by introducing a demographic attributes balancing mechanism in generated training datasets.
  arXiv  Detail & Related papers  (2024-06-24T12:33:21Z)
• Massively Annotated Datasets for Assessment of Synthetic and Real Data in Face Recognition [0.2775636978045794]
  We study the drift between the performance of models trained on real and synthetic datasets. We conduct studies on the differences between real and synthetic datasets on the attribute set. Interestingly enough, we have verified that while real samples suffice to explain the synthetic distribution, the opposite could not be further from being true.
  arXiv  Detail & Related papers  (2024-04-23T17:10:49Z)
• If It's Not Enough, Make It So: Reducing Authentic Data Demand in Face Recognition through Synthetic Faces [16.977459035497162]
  Large face datasets are primarily sourced from web-based images, lacking explicit user consent. In this paper, we examine whether and how synthetic face data can be used to train effective face recognition models.
  arXiv  Detail & Related papers  (2024-04-04T15:45:25Z)
• Bias and Diversity in Synthetic-based Face Recognition [12.408456748469426]
  We investigate how the diversity of synthetic face recognition datasets compares to authentic datasets. We look at the distribution of gender, ethnicity, age, and head position. With regard to bias, it can be seen that the synthetic-based models share a similar bias behavior with the authentic-based models.
  arXiv  Detail & Related papers  (2023-11-07T13:12:34Z)
• On the Stability of Iterative Retraining of Generative Models on their own Data [56.153542044045224]
  We study the impact of training generative models on mixed datasets. We first prove the stability of iterative training under the condition that the initial generative models approximate the data distribution well enough. We empirically validate our theory on both synthetic and natural images by iteratively training normalizing flows and state-of-the-art diffusion models.
  arXiv  Detail & Related papers  (2023-09-30T16:41:04Z)
• Face Recognition Using Synthetic Face Data [0.0]
  We highlight the promising application of synthetic data, generated through rendering digital faces via our computer graphics pipeline, in achieving competitive results. By finetuning the model,we obtain results that rival those achieved when training with hundreds of thousands of real images. We also investigate the contribution of adding intra-class variance factors (e.g., makeup, accessories, haircuts) on model performance.
  arXiv  Detail & Related papers  (2023-05-17T09:26:10Z)
• CAFE: Learning to Condense Dataset by Aligning Features [72.99394941348757]
  We propose a novel scheme to Condense dataset by Aligning FEatures (CAFE) At the heart of our approach is an effective strategy to align features from the real and synthetic data across various scales. We validate the proposed CAFE across various datasets, and demonstrate that it generally outperforms the state of the art.
  arXiv  Detail & Related papers  (2022-03-03T05:58:49Z)
• Transitioning from Real to Synthetic data: Quantifying the bias in model [1.6134566438137665]
  This study aims to establish a trade-off between bias and fairness in the models trained using synthetic data. We demonstrate there exist a varying levels of bias impact on models trained using synthetic data.
  arXiv  Detail & Related papers  (2021-05-10T06:57:14Z)
• Representative & Fair Synthetic Data [68.8204255655161]
  We present a framework to incorporate fairness constraints into the self-supervised learning process. We generate a representative as well as fair version of the UCI Adult census data set. We consider representative & fair synthetic data a promising future building block to teach algorithms not on historic worlds, but rather on the worlds that we strive to live in.
  arXiv  Detail & Related papers  (2021-04-07T09:19:46Z)
• Enhancing Facial Data Diversity with Style-based Face Aging [59.984134070735934]
  In particular, face datasets are typically biased in terms of attributes such as gender, age, and race. We propose a novel, generative style-based architecture for data augmentation that captures fine-grained aging patterns. We show that the proposed method outperforms state-of-the-art algorithms for age transfer.
  arXiv  Detail & Related papers  (2020-06-06T21:53:44Z)
• Long-Tailed Recognition Using Class-Balanced Experts [128.73438243408393]
  We propose an ensemble of class-balanced experts that combines the strength of diverse classifiers. Our ensemble of class-balanced experts reaches results close to state-of-the-art and an extended ensemble establishes a new state-of-the-art on two benchmarks for long-tailed recognition.
  arXiv  Detail & Related papers  (2020-04-07T20:57:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.