Analyzing Cultural Representations of Emotions in LLMs through Mixed Emotion Survey

• URL: http://arxiv.org/abs/2408.02143v1
• Date: Sun, 4 Aug 2024 20:56:05 GMT
• Title: Analyzing Cultural Representations of Emotions in LLMs through Mixed Emotion Survey
• Authors: Shiran Dudy, Ibrahim Said Ahmad, Ryoko Kitajima, Agata Lapedriza,
• Abstract summary: This study focuses on analyzing the cultural representations of emotions in Large Language Models (LLMs) Our methodology is based on the studies of Miyamoto et al. (2010), which identified distinctive emotional indicators in Japanese and American human responses. We find that models have limited alignment with the evidence in the literature.
• Score: 2.9213203896291766
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large Language Models (LLMs) have gained widespread global adoption, showcasing advanced linguistic capabilities across multiple of languages. There is a growing interest in academia to use these models to simulate and study human behaviors. However, it is crucial to acknowledge that an LLM's proficiency in a specific language might not fully encapsulate the norms and values associated with its culture. Concerns have emerged regarding potential biases towards Anglo-centric cultures and values due to the predominance of Western and US-based training data. This study focuses on analyzing the cultural representations of emotions in LLMs, in the specific case of mixed-emotion situations. Our methodology is based on the studies of Miyamoto et al. (2010), which identified distinctive emotional indicators in Japanese and American human responses. We first administer their mixed emotion survey to five different LLMs and analyze their outputs. Second, we experiment with contextual variables to explore variations in responses considering both language and speaker origin. Thirdly, we expand our investigation to encompass additional East Asian and Western European origin languages to gauge their alignment with their respective cultures, anticipating a closer fit. We find that (1) models have limited alignment with the evidence in the literature; (2) written language has greater effect on LLMs' response than information on participants origin; and (3) LLMs responses were found more similar for East Asian languages than Western European languages.

Related papers

• Multi-ToM: Evaluating Multilingual Theory of Mind Capabilities in Large Language Models [3.9532244541907793]
  Theory of Mind (ToM) refers to the cognitive ability to infer and attribute mental states to oneself and others. It remains unclear to what extent large language models (LLMs) demonstrate ToM across diverse languages and cultural contexts. This paper introduces a comprehensive study of multilingual ToM capabilities aimed at addressing this gap.
  arXiv  Detail & Related papers  (2024-11-24T22:37:59Z)
• Large Language Models Reflect the Ideology of their Creators [73.25935570218375]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language. We uncover notable diversity in the ideological stance exhibited across different LLMs and languages.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• See It from My Perspective: Diagnosing the Western Cultural Bias of Large Vision-Language Models in Image Understanding [78.88461026069862]
  Vision-language models (VLMs) can respond to queries about images in many languages. We present a novel investigation that demonstrates and localizes Western bias in image understanding.
  arXiv  Detail & Related papers  (2024-06-17T15:49:51Z)
• Cultural Value Differences of LLMs: Prompt, Language, and Model Size [35.176429953825924]
  Our study aims to identify behavior patterns in cultural values exhibited by large language models (LLMs) The studied variants include question ordering, prompting language, and model size. Our experiments reveal that query language and model size of LLM are the main factors resulting in cultural value differences.
  arXiv  Detail & Related papers  (2024-06-17T12:35:33Z)
• Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsense [98.09670425244462]
  Large language models (LLMs) have demonstrated substantial commonsense understanding. This paper examines the capabilities and limitations of several state-of-the-art LLMs in the context of cultural commonsense tasks.
  arXiv  Detail & Related papers  (2024-05-07T20:28:34Z)
• CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Prompting [73.94059188347582]
  We uncover culture perceptions of three SOTA models on 110 countries and regions on 8 culture-related topics through culture-conditioned generations. We discover that culture-conditioned generation consist of linguistic "markers" that distinguish marginalized cultures apart from default cultures.
  arXiv  Detail & Related papers  (2024-04-16T00:50:43Z)
• Does Mapo Tofu Contain Coffee? Probing LLMs for Food-related Cultural Knowledge [47.57055368312541]
  We introduce FmLAMA, a multilingual dataset centered on food-related cultural facts and variations in food practices. We analyze LLMs across various architectures and configurations, evaluating their performance in both monolingual and multilingual settings.
  arXiv  Detail & Related papers  (2024-04-10T08:49:27Z)
• Investigating Cultural Alignment of Large Language Models [10.738300803676655]
  We show that Large Language Models (LLMs) genuinely encapsulate the diverse knowledge adopted by different cultures. We quantify cultural alignment by simulating sociological surveys, comparing model responses to those of actual survey participants as references. We introduce Anthropological Prompting, a novel method leveraging anthropological reasoning to enhance cultural alignment.
  arXiv  Detail & Related papers  (2024-02-20T18:47:28Z)
• Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models [89.94270049334479]
  This paper identifies a cultural dominance issue within large language models (LLMs) LLMs often provide inappropriate English-culture-related answers that are not relevant to the expected culture when users ask in non-English languages.
  arXiv  Detail & Related papers  (2023-10-19T05:38:23Z)
• Multilingual Language Models are not Multicultural: A Case Study in Emotion [8.73324795579955]
  We investigate whether the widely-used multilingual LMs in 2023 reflect differences in emotional expressions across cultures and languages. We find that embeddings obtained from LMs are Anglocentric, and generative LMs reflect Western norms, even when responding to prompts in other languages.
  arXiv  Detail & Related papers  (2023-07-03T21:54:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.