Speaking Multiple Languages Affects the Moral Bias of Language Models

• URL: http://arxiv.org/abs/2211.07733v2
• Date: Thu, 1 Jun 2023 08:41:34 GMT
• Title: Speaking Multiple Languages Affects the Moral Bias of Language Models
• Authors: Katharina H\"ammerl, Bj\"orn Deiseroth, Patrick Schramowski, Jind\v{r}ich Libovick\'y, Constantin A. Rothkopf, Alexander Fraser, Kristian Kersting
• Abstract summary: Pre-trained multilingual language models (PMLMs) are commonly used when dealing with data from multiple languages and cross-lingual transfer. Do the models capture moral norms from English and impose them on other languages? Our experiments demonstrate that, indeed, PMLMs encode differing moral biases, but these do not necessarily correspond to cultural differences or commonalities in human opinions.
• Score: 70.94372902010232
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pre-trained multilingual language models (PMLMs) are commonly used when dealing with data from multiple languages and cross-lingual transfer. However, PMLMs are trained on varying amounts of data for each language. In practice this means their performance is often much better on English than many other languages. We explore to what extent this also applies to moral norms. Do the models capture moral norms from English and impose them on other languages? Do the models exhibit random and thus potentially harmful beliefs in certain languages? Both these issues could negatively impact cross-lingual transfer and potentially lead to harmful outcomes. In this paper, we (1) apply the MoralDirection framework to multilingual models, comparing results in German, Czech, Arabic, Chinese, and English, (2) analyse model behaviour on filtered parallel subtitles corpora, and (3) apply the models to a Moral Foundations Questionnaire, comparing with human responses from different countries. Our experiments demonstrate that, indeed, PMLMs encode differing moral biases, but these do not necessarily correspond to cultural differences or commonalities in human opinions. We release our code and models.

Related papers

• Decoding Multilingual Moral Preferences: Unveiling LLM's Biases Through the Moral Machine Experiment [11.82100047858478]
  This paper builds on the moral machine experiment (MME) to investigate the moral preferences of five large language models in a multilingual setting. We generate 6500 scenarios of the MME and prompt the models in ten languages on which action to take. Our analysis reveals that all LLMs inhibit different moral biases to some degree and that they not only differ from the human preferences but also across multiple languages within the models themselves.
  arXiv  Detail & Related papers  (2024-07-21T14:48:13Z)
• Does Cross-Cultural Alignment Change the Commonsense Morality of Language Models? [3.48097307252416]
  alignment of language models with human preferences is a common approach to making a language model useful to end users. Most alignment work is done in English, and human preference datasets are dominated by English. We investigate whether aligning Japanese language models with English resources marginalizes the preference of non-English speaking users.
  arXiv  Detail & Related papers  (2024-06-24T04:50:12Z)
• Do Moral Judgment and Reasoning Capability of LLMs Change with Language? A Study using the Multilingual Defining Issues Test [21.108525674360898]
  We extend the work of beyond English to five new languages (Chinese, Hindi, Russian, Spanish and Swahili) Our study shows that the moral reasoning ability for all models, as indicated by the post-conventional score, is substantially inferior for Hindi and Swahili, compared to Spanish, Russian, Chinese and English.
  arXiv  Detail & Related papers  (2024-02-03T12:52:36Z)
• Multilingual Text-to-Image Generation Magnifies Gender Stereotypes and Prompt Engineering May Not Help You [64.74707085021858]
  We show that multilingual models suffer from significant gender biases just as monolingual models do. We propose a novel benchmark, MAGBIG, intended to foster research on gender bias in multilingual models. Our results show that not only do models exhibit strong gender biases but they also behave differently across languages.
  arXiv  Detail & Related papers  (2024-01-29T12:02:28Z)
• Knowledge of cultural moral norms in large language models [3.475552182166427]
  We investigate the extent to which monolingual English language models contain knowledge about moral norms in different countries. We perform our analyses with two public datasets from the World Values Survey and PEW global surveys on morality. We find that pre-trained English language models predict empirical moral norms across countries worse than the English moral norms reported previously.
  arXiv  Detail & Related papers  (2023-06-02T18:23:35Z)
• Language Model Tokenizers Introduce Unfairness Between Languages [98.92630681729518]
  We show how disparity in the treatment of different languages arises at the tokenization stage, well before a model is even invoked. Character-level and byte-level models also exhibit over 4 times the difference in the encoding length for some language pairs. We make the case that we should train future language models using multilingually fair subword tokenizers.
  arXiv  Detail & Related papers  (2023-05-17T14:17:57Z)
• Do Multilingual Language Models Capture Differing Moral Norms? [71.52261949766101]
  Massively multilingual sentence representations are trained on large corpora of uncurated data. This may cause the models to grasp cultural values including moral judgments from the high-resource languages. The lack of data in certain languages can also lead to developing random and thus potentially harmful beliefs.
  arXiv  Detail & Related papers  (2022-03-18T12:26:37Z)
• Discovering Representation Sprachbund For Multilingual Pre-Training [139.05668687865688]
  We generate language representation from multilingual pre-trained models and conduct linguistic analysis. We cluster all the target languages into multiple groups and name each group as a representation sprachbund. Experiments are conducted on cross-lingual benchmarks and significant improvements are achieved compared to strong baselines.
  arXiv  Detail & Related papers  (2021-09-01T09:32:06Z)
• XPersona: Evaluating Multilingual Personalized Chatbot [76.00426517401894]
  We propose a multi-lingual extension of Persona-Chat, namely XPersona. Our dataset includes persona conversations in six different languages other than English for building and evaluating multilingual personalized agents.
  arXiv  Detail & Related papers  (2020-03-17T07:52:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.