Towards Intercultural Affect Recognition: Audio-Visual Affect Recognition in the Wild Across Six Cultures

• URL: http://arxiv.org/abs/2208.00344v1
• Date: Sun, 31 Jul 2022 02:39:17 GMT
• Title: Towards Intercultural Affect Recognition: Audio-Visual Affect Recognition in the Wild Across Six Cultures
• Authors: Leena Mathur, Ralph Adolphs, Maja J Matari\'c
• Abstract summary: We present the first systematic study of intercultural affect recognition models using videos of real-world dyadic interactions from six cultures. Across all six cultures, our findings demonstrate that intercultural affect recognition models were as effective or more effective than intracultural models. Our paper presents a proof-of-concept and motivation for the future development of intercultural affect recognition systems.
• Score: 15.37706529432381
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In our multicultural world, affect-aware AI systems that support humans need the ability to perceive affect across variations in emotion expression patterns across cultures. These models must perform well in cultural contexts on which they have not been trained. A standard assumption in affective computing is that affect recognition models trained and used within the same culture (intracultural) will perform better than models trained on one culture and used on different cultures (intercultural). We test this assumption and present the first systematic study of intercultural affect recognition models using videos of real-world dyadic interactions from six cultures. We develop an attention-based feature selection approach under temporal causal discovery to identify behavioral cues that can be leveraged in intercultural affect recognition models. Across all six cultures, our findings demonstrate that intercultural affect recognition models were as effective or more effective than intracultural models. We identify and contribute useful behavioral features for intercultural affect recognition; facial features from the visual modality were more useful than the audio modality in this study's context. Our paper presents a proof-of-concept and motivation for the future development of intercultural affect recognition systems.

Related papers

• CROPE: Evaluating In-Context Adaptation of Vision and Language Models to Culture-Specific Concepts [45.77570690529597]
  We introduce CROPE, a visual question answering benchmark designed to probe the knowledge of culture-specific concepts. Our evaluation of several state-of-the-art open Vision and Language models shows large performance disparities between culture-specific and common concepts. Experiments with contextual knowledge indicate that models struggle to effectively utilize multimodal information and bind culture-specific concepts to their depictions.
  arXiv  Detail & Related papers  (2024-10-20T17:31:19Z)
• Navigating the Cultural Kaleidoscope: A Hitchhiker's Guide to Sensitivity in Large Language Models [4.771099208181585]
  LLMs are increasingly deployed in global applications, ensuring users from diverse backgrounds feel respected and understood. Cultural harm can arise when these models fail to align with specific cultural norms, resulting in misrepresentations or violations of cultural values. We present two key contributions: A cultural harm test dataset, created to assess model outputs across different cultural contexts through scenarios that expose potential cultural insensitivities, and a culturally aligned preference dataset, aimed at restoring cultural sensitivity through fine-tuning based on feedback from diverse annotators.
  arXiv  Detail & Related papers  (2024-10-15T18:13:10Z)
• From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models [10.121734731147376]
  Vision-language models' performance remains suboptimal on images from non-western cultures. Various benchmarks have been proposed to test models' cultural inclusivity, but they have limited coverage of cultures. We introduce the GlobalRG benchmark, comprising two challenging tasks: retrieval across universals and cultural visual grounding.
  arXiv  Detail & Related papers  (2024-06-28T23:28:28Z)
• Extrinsic Evaluation of Cultural Competence in Large Language Models [53.626808086522985]
  We focus on extrinsic evaluation of cultural competence in two text generation tasks. We evaluate model outputs when an explicit cue of culture, specifically nationality, is perturbed in the prompts. We find weak correlations between text similarity of outputs for different countries and the cultural values of these countries.
  arXiv  Detail & Related papers  (2024-06-17T14:03:27Z)
• CulturePark: Boosting Cross-cultural Understanding in Large Language Models [63.452948673344395]
  This paper introduces CulturePark, an LLM-powered multi-agent communication framework for cultural data collection. It generates high-quality cross-cultural dialogues encapsulating human beliefs, norms, and customs. We evaluate these models across three downstream tasks: content moderation, cultural alignment, and cultural education.
  arXiv  Detail & Related papers  (2024-05-24T01:49:02Z)
• What You Use is What You Get: Unforced Errors in Studying Cultural Aspects in Agile Software Development [2.9418191027447906]
  Investigating the influence of cultural characteristics is challenging due to the multi-faceted concept of culture. Cultural and social aspects are of high importance for their successful use in practice.
  arXiv  Detail & Related papers  (2024-04-25T20:08:37Z)
• CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Prompting [73.94059188347582]
  We uncover culture perceptions of three SOTA models on 110 countries and regions on 8 culture-related topics through culture-conditioned generations. We discover that culture-conditioned generation consist of linguistic "markers" that distinguish marginalized cultures apart from default cultures.
  arXiv  Detail & Related papers  (2024-04-16T00:50:43Z)
• Massively Multi-Cultural Knowledge Acquisition & LM Benchmarking [48.21982147529661]
  This paper introduces a novel approach for massively multicultural knowledge acquisition. Our method strategically navigates from densely informative Wikipedia documents on cultural topics to an extensive network of linked pages. Our work marks an important step towards deeper understanding and bridging the gaps of cultural disparities in AI.
  arXiv  Detail & Related papers  (2024-02-14T18:16:54Z)
• Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models [89.94270049334479]
  This paper identifies a cultural dominance issue within large language models (LLMs) LLMs often provide inappropriate English-culture-related answers that are not relevant to the expected culture when users ask in non-English languages.
  arXiv  Detail & Related papers  (2023-10-19T05:38:23Z)
• Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study [9.919972416590124]
  ChatGPT has garnered widespread recognition for its exceptional ability to generate human-like responses in dialogue. We investigate the underlying cultural background of ChatGPT by analyzing its responses to questions designed to quantify human cultural differences.
  arXiv  Detail & Related papers  (2023-03-30T15:43:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.