Decoding Multilingual Moral Preferences: Unveiling LLM's Biases Through the Moral Machine Experiment

• URL: http://arxiv.org/abs/2407.15184v1
• Date: Sun, 21 Jul 2024 14:48:13 GMT
• Title: Decoding Multilingual Moral Preferences: Unveiling LLM's Biases Through the Moral Machine Experiment
• Authors: Karina Vida, Fabian Damken, Anne Lauscher,
• Abstract summary: This paper builds on the moral machine experiment (MME) to investigate the moral preferences of five large language models in a multilingual setting. We generate 6500 scenarios of the MME and prompt the models in ten languages on which action to take. Our analysis reveals that all LLMs inhibit different moral biases to some degree and that they not only differ from the human preferences but also across multiple languages within the models themselves.
• Score: 11.82100047858478
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) increasingly find their way into the most diverse areas of our everyday lives. They indirectly influence people's decisions or opinions through their daily use. Therefore, understanding how and which moral judgements these LLMs make is crucial. However, morality is not universal and depends on the cultural background. This raises the question of whether these cultural preferences are also reflected in LLMs when prompted in different languages or whether moral decision-making is consistent across different languages. So far, most research has focused on investigating the inherent values of LLMs in English. While a few works conduct multilingual analyses of moral bias in LLMs in a multilingual setting, these analyses do not go beyond atomic actions. To the best of our knowledge, a multilingual analysis of moral bias in dilemmas has not yet been conducted. To address this, our paper builds on the moral machine experiment (MME) to investigate the moral preferences of five LLMs, Falcon, Gemini, Llama, GPT, and MPT, in a multilingual setting and compares them with the preferences collected from humans belonging to different cultures. To accomplish this, we generate 6500 scenarios of the MME and prompt the models in ten languages on which action to take. Our analysis reveals that all LLMs inhibit different moral biases to some degree and that they not only differ from the human preferences but also across multiple languages within the models themselves. Moreover, we find that almost all models, particularly Llama 3, divert greatly from human values and, for instance, prefer saving fewer people over saving more.

Related papers

• Large Language Models Reflect the Ideology of their Creators [73.25935570218375]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language. We uncover notable diversity in the ideological stance exhibited across different LLMs and languages.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• Language Model Alignment in Multilingual Trolley Problems [138.5684081822807]
  Building on the Moral Machine experiment, we develop a cross-lingual corpus of moral dilemma vignettes in over 100 languages called MultiTP. Our analysis explores the alignment of 19 different LLMs with human judgments, capturing preferences across six moral dimensions. We discover significant variance in alignment across languages, challenging the assumption of uniform moral reasoning in AI systems.
  arXiv  Detail & Related papers  (2024-07-02T14:02:53Z)
• Exploring and steering the moral compass of Large Language Models [55.2480439325792]
  Large Language Models (LLMs) have become central to advancing automation and decision-making across various sectors. This study proposes a comprehensive comparative analysis of the most advanced LLMs to assess their moral profiles.
  arXiv  Detail & Related papers  (2024-05-27T16:49:22Z)
• Ethical Reasoning and Moral Value Alignment of LLMs Depend on the Language we Prompt them in [19.675262411557235]
  This paper explores how three prominent LLMs -- GPT-4, ChatGPT, and Llama2-70B-Chat -- perform ethical reasoning in different languages. We experiment with six languages: English, Spanish, Russian, Chinese, Hindi, and Swahili. We find that GPT-4 is the most consistent and unbiased ethical reasoner across languages, while ChatGPT and Llama2-70B-Chat show significant moral value bias when we move to languages other than English.
  arXiv  Detail & Related papers  (2024-04-29T06:42:27Z)
• Moral Foundations of Large Language Models [6.6445242437134455]
  Moral foundations theory (MFT) is a psychological assessment tool that decomposes human moral reasoning into five factors. As large language models (LLMs) are trained on datasets collected from the internet, they may reflect the biases that are present in such corpora. This paper uses MFT as a lens to analyze whether popular LLMs have acquired a bias towards a particular set of moral values.
  arXiv  Detail & Related papers  (2023-10-23T20:05:37Z)
• Are Multilingual LLMs Culturally-Diverse Reasoners? An Investigation into Multicultural Proverbs and Sayings [73.48336898620518]
  Large language models (LLMs) are highly adept at question answering and reasoning tasks. We study the ability of a wide range of state-of-the-art multilingual LLMs to reason with proverbs and sayings in a conversational context.
  arXiv  Detail & Related papers  (2023-09-15T17:45:28Z)
• Speaking Multiple Languages Affects the Moral Bias of Language Models [70.94372902010232]
  Pre-trained multilingual language models (PMLMs) are commonly used when dealing with data from multiple languages and cross-lingual transfer. Do the models capture moral norms from English and impose them on other languages? Our experiments demonstrate that, indeed, PMLMs encode differing moral biases, but these do not necessarily correspond to cultural differences or commonalities in human opinions.
  arXiv  Detail & Related papers  (2022-11-14T20:08:54Z)
• Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity [0.0]
  This study investigates whether Large Language Models reproduce the moral biases associated with political groups in the United States. Using tools from Moral Foundations Theory, it is shown that these LLMs are indeed moral mimics.
  arXiv  Detail & Related papers  (2022-09-24T23:55:53Z)
• Do Multilingual Language Models Capture Differing Moral Norms? [71.52261949766101]
  Massively multilingual sentence representations are trained on large corpora of uncurated data. This may cause the models to grasp cultural values including moral judgments from the high-resource languages. The lack of data in certain languages can also lead to developing random and thus potentially harmful beliefs.
  arXiv  Detail & Related papers  (2022-03-18T12:26:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.