Cultural Bias in Large Language Models: Evaluating AI Agents through Moral Questionnaires

• URL: http://arxiv.org/abs/2507.10073v2
• Date: Thu, 31 Jul 2025 03:13:46 GMT
• Title: Cultural Bias in Large Language Models: Evaluating AI Agents through Moral Questionnaires
• Authors: Simon Münker,
• Abstract summary: Large Language Models fail to represent diverse cultural moral frameworks despite their linguistic capabilities.<n>Surprisingly, increased model size doesn't consistently improve cultural representation fidelity.<n>Our results call for more grounded alignment objectives and evaluation metrics to ensure AI systems represent diverse human values.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Are AI systems truly representing human values, or merely averaging across them? Our study suggests a concerning reality: Large Language Models (LLMs) fail to represent diverse cultural moral frameworks despite their linguistic capabilities. We expose significant gaps between AI-generated and human moral intuitions by applying the Moral Foundations Questionnaire across 19 cultural contexts. Comparing multiple state-of-the-art LLMs' origins against human baseline data, we find these models systematically homogenize moral diversity. Surprisingly, increased model size doesn't consistently improve cultural representation fidelity. Our findings challenge the growing use of LLMs as synthetic populations in social science research and highlight a fundamental limitation in current AI alignment approaches. Without data-driven alignment beyond prompting, these systems cannot capture the nuanced, culturally-specific moral intuitions. Our results call for more grounded alignment objectives and evaluation metrics to ensure AI systems represent diverse human values rather than flattening the moral landscape.

Related papers

• Do Large Language Models Understand Morality Across Cultures? [0.5356944479760104]
  This study investigates the extent to which large language models capture cross-cultural differences and similarities in moral perspectives.<n>Our results reveal that current LLMs often fail to reproduce the full spectrum of cross-cultural moral variation.<n>These findings highlight a pressing need for more robust approaches to mitigate biases and improve cultural representativeness in LLMs.
  arXiv  Detail & Related papers  (2025-07-28T20:25:36Z)
• Whose Morality Do They Speak? Unraveling Cultural Bias in Multilingual Language Models [0.0]
  Large language models (LLMs) have become integral tools in diverse domains, yet their moral reasoning capabilities remain underexplored.<n>This study investigates whether multilingual LLMs, such as GPT-3.5-Turbo, reflect culturally specific moral values or impose dominant moral norms.<n>Using the updated Moral Foundations Questionnaire (MFQ-2) in eight languages, the study analyzes the models' adherence to six core moral foundations.
  arXiv  Detail & Related papers  (2024-12-25T10:17:15Z)
• Large Language Models as Mirrors of Societal Moral Standards [0.5852077003870417]
  Language models can, to a limited extent, represent moral norms in a variety of cultural contexts.<n>This study evaluates the effectiveness of these models using information from two surveys, the WVS and the PEW, that encompass moral perspectives from over 40 countries.<n>The results show that biases exist in both monolingual and multilingual models, and they typically fall short of accurately capturing the moral intricacies of diverse cultures.
  arXiv  Detail & Related papers  (2024-12-01T20:20:35Z)
• Large Language Models Reflect the Ideology of their Creators [71.65505524599888]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language.<n>This paper shows that the ideological stance of an LLM appears to reflect the worldview of its creators.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• Language Model Alignment in Multilingual Trolley Problems [138.5684081822807]
  Building on the Moral Machine experiment, we develop a cross-lingual corpus of moral dilemma vignettes in over 100 languages called MultiTP.<n>Our analysis explores the alignment of 19 different LLMs with human judgments, capturing preferences across six moral dimensions.<n>We discover significant variance in alignment across languages, challenging the assumption of uniform moral reasoning in AI systems.
  arXiv  Detail & Related papers  (2024-07-02T14:02:53Z)
• Culturally-Attuned Moral Machines: Implicit Learning of Human Value Systems by AI through Inverse Reinforcement Learning [11.948092546676687]
  We argue that the value system of an AI should be culturally attuned. How AI systems might acquire such codes from human observation and interaction has remained an open question. We show that an AI agent learning from the average behavior of a particular cultural group can acquire altruistic characteristics reflective of that group's behavior.
  arXiv  Detail & Related papers  (2023-12-29T05:39:10Z)
• Learning Human-like Representations to Enable Learning Human Values [11.236150405125754]
  We explore the effects of representational alignment between humans and AI agents on learning human values. We show that this kind of representational alignment can support safely learning and exploring human values in the context of personalization.
  arXiv  Detail & Related papers  (2023-12-21T18:31:33Z)
• Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties [68.66719970507273]
  Value pluralism is the view that multiple correct values may be held in tension with one another. As statistical learners, AI systems fit to averages by default, washing out potentially irreducible value conflicts. We introduce ValuePrism, a large-scale dataset of 218k values, rights, and duties connected to 31k human-written situations.
  arXiv  Detail & Related papers  (2023-09-02T01:24:59Z)
• Cultural Incongruencies in Artificial Intelligence [5.817158625734485]
  We describe a set of cultural dependencies and incongruencies in the context of AI-based language and vision technologies. Problems arise when these technologies interact with globally diverse societies and cultures, with different values and interpretive practices.
  arXiv  Detail & Related papers  (2022-11-19T18:45:02Z)
• Metaethical Perspectives on 'Benchmarking' AI Ethics [81.65697003067841]
  Benchmarks are seen as the cornerstone for measuring technical progress in Artificial Intelligence (AI) research. An increasingly prominent research area in AI is ethics, which currently has no set of benchmarks nor commonly accepted way for measuring the 'ethicality' of an AI system. We argue that it makes more sense to talk about 'values' rather than 'ethics' when considering the possible actions of present and future AI systems.
  arXiv  Detail & Related papers  (2022-04-11T14:36:39Z)
• Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotes [72.64975113835018]
  Motivated by descriptive ethics, we investigate a novel, data-driven approach to machine ethics. We introduce Scruples, the first large-scale dataset with 625,000 ethical judgments over 32,000 real-life anecdotes. Our dataset presents a major challenge to state-of-the-art neural language models, leaving significant room for improvement.
  arXiv  Detail & Related papers  (2020-08-20T17:34:15Z)
• Aligning AI With Shared Human Values [85.2824609130584]
  We introduce the ETHICS dataset, a new benchmark that spans concepts in justice, well-being, duties, virtues, and commonsense morality. We find that current language models have a promising but incomplete ability to predict basic human ethical judgements. Our work shows that progress can be made on machine ethics today, and it provides a steppingstone toward AI that is aligned with human values.
  arXiv  Detail & Related papers  (2020-08-05T17:59:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.