Mitigation of gender bias in automatic facial non-verbal behaviors generation

• URL: http://arxiv.org/abs/2410.07274v1
• Date: Wed, 9 Oct 2024 06:41:24 GMT
• Title: Mitigation of gender bias in automatic facial non-verbal behaviors generation
• Authors: Alice Delbosc, Magalie Ochs, Nicolas Sabouret, Brian Ravenet, Stephane Ayache,
• Abstract summary: We introduce a classifier capable of discerning the gender of a speaker from their non-verbal cues. We present a new model, FairGenderGen, which integrates a gender discriminator and a gradient reversal layer into our previous behavior generation model.
• Score: 0.45088680687810573
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Research on non-verbal behavior generation for social interactive agents focuses mainly on the believability and synchronization of non-verbal cues with speech. However, existing models, predominantly based on deep learning architectures, often perpetuate biases inherent in the training data. This raises ethical concerns, depending on the intended application of these agents. This paper addresses these issues by first examining the influence of gender on facial non-verbal behaviors. We concentrate on gaze, head movements, and facial expressions. We introduce a classifier capable of discerning the gender of a speaker from their non-verbal cues. This classifier achieves high accuracy on both real behavior data, extracted using state-of-the-art tools, and synthetic data, generated from a model developed in previous work.Building upon this work, we present a new model, FairGenderGen, which integrates a gender discriminator and a gradient reversal layer into our previous behavior generation model. This new model generates facial non-verbal behaviors from speech features, mitigating gender sensitivity in the generated behaviors. Our experiments demonstrate that the classifier, developed in the initial phase, is no longer effective in distinguishing the gender of the speaker from the generated non-verbal behaviors.

Related papers

• Gender Bias in Instruction-Guided Speech Synthesis Models [55.2480439325792]
  This study investigates the potential gender bias in how models interpret occupation-related prompts. We explore whether these models exhibit tendencies to amplify gender stereotypes when interpreting such prompts. Our experimental results reveal the model's tendency to exhibit gender bias for certain occupations.
  arXiv  Detail & Related papers  (2025-02-08T17:38:24Z)
• Nonverbal Interaction Detection [83.40522919429337]
  This work addresses a new challenge of understanding human nonverbal interaction in social contexts. We contribute a novel large-scale dataset, called NVI, which is meticulously annotated to include bounding boxes for humans and corresponding social groups. Second, we establish a new task NVI-DET for nonverbal interaction detection, which is formalized as identifying triplets in the form individual, group, interaction> from images. Third, we propose a nonverbal interaction detection hypergraph (NVI-DEHR), a new approach that explicitly models high-order nonverbal interactions using hypergraphs.
  arXiv  Detail & Related papers  (2024-07-11T02:14:06Z)
• Exploring Gender Biases in Language Patterns of Human-Conversational Agent Conversations [1.0878040851638]
  This research aims to delve deeper into the impacts of gender biases in human-CA interactions. It aims to understand how pre-existing gender biases might be triggered by CAs' gender designs. The findings aim to inform ethical design of conversational agents, addressing whether gender assignment in CAs is appropriate.
  arXiv  Detail & Related papers  (2024-01-05T19:11:17Z)
• Will the Prince Get True Love's Kiss? On the Model Sensitivity to Gender Perturbation over Fairytale Texts [87.62403265382734]
  Recent studies show that traditional fairytales are rife with harmful gender biases. This work aims to assess learned biases of language models by evaluating their robustness against gender perturbations.
  arXiv  Detail & Related papers  (2023-10-16T22:25:09Z)
• Towards the generation of synchronized and believable non-verbal facial behaviors of a talking virtual agent [0.0]
  This paper introduces a new model to generate rhythmically relevant non-verbal facial behaviors for virtual agents while they speak. We find that training the model with two different sets of data, instead of one, did not necessarily improve its performance. We also show that employing an adversarial model, in which fabricated fake examples are introduced during the training phase, increases the perception of synchronization with speech.
  arXiv  Detail & Related papers  (2023-09-15T07:25:36Z)
• How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions? [67.97752431429865]
  We study the effect on the diversity of the generated images when adding ethical intervention. Preliminary studies indicate that a large change in the model predictions is triggered by certain phrases such as 'irrespective of gender'
  arXiv  Detail & Related papers  (2022-10-27T07:32:39Z)
• Gender Stereotyping Impact in Facial Expression Recognition [1.5340540198612824]
  In recent years, machine learning-based models have become the most popular approach to Facial Expression Recognition (FER) In publicly available FER datasets, apparent gender representation is usually mostly balanced, but their representation in the individual label is not. We generate derivative datasets with different amounts of stereotypical bias by altering the gender proportions of certain labels. We observe a discrepancy in the recognition of certain emotions between genders of up to $29 %$ under the worst bias conditions.
  arXiv  Detail & Related papers  (2022-10-11T10:52:23Z)
• CIAO! A Contrastive Adaptation Mechanism for Non-Universal Facial Expression Recognition [80.07590100872548]
  We propose Contrastive Inhibitory Adaptati On (CIAO), a mechanism that adapts the last layer of facial encoders to depict specific affective characteristics on different datasets. CIAO presents an improvement in facial expression recognition performance over six different datasets with very unique affective representations.
  arXiv  Detail & Related papers  (2022-08-10T15:46:05Z)
• Are Commercial Face Detection Models as Biased as Academic Models? [64.71318433419636]
  We compare academic and commercial face detection systems, specifically examining robustness to noise. We find that state-of-the-art academic face detection models exhibit demographic disparities in their noise robustness. We conclude that commercial models are always as biased or more biased than an academic model.
  arXiv  Detail & Related papers  (2022-01-25T02:21:42Z)
• Learning Graph Representation of Person-specific Cognitive Processes from Audio-visual Behaviours for Automatic Personality Recognition [17.428626029689653]
  We propose to represent the target subjects person-specific cognition in the form of a person-specific CNN architecture. Each person-specific CNN is explored by the Neural Architecture Search (NAS) and a novel adaptive loss function. Experimental results show that the produced graph representations are well associated with target subjects' personality traits.
  arXiv  Detail & Related papers  (2021-10-26T11:04:23Z)
• Responsible AI: Gender bias assessment in emotion recognition [6.833826997240138]
  This research work aims to study a gender bias in deep learning methods for facial expression recognition. More biased neural networks show bigger accuracy gap in emotion recognition between male and female test sets.
  arXiv  Detail & Related papers  (2021-03-21T17:00:21Z)
• Multi-Dimensional Gender Bias Classification [67.65551687580552]
  Machine learning models can inadvertently learn socially undesirable patterns when training on gender biased text. We propose a general framework that decomposes gender bias in text along several pragmatic and semantic dimensions. Using this fine-grained framework, we automatically annotate eight large scale datasets with gender information.
  arXiv  Detail & Related papers  (2020-05-01T21:23:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.