Knowledge of cultural moral norms in large language models
- URL: http://arxiv.org/abs/2306.01857v1
- Date: Fri, 2 Jun 2023 18:23:35 GMT
- Title: Knowledge of cultural moral norms in large language models
- Authors: Aida Ramezani, Yang Xu
- Abstract summary: We investigate the extent to which monolingual English language models contain knowledge about moral norms in different countries.
We perform our analyses with two public datasets from the World Values Survey and PEW global surveys on morality.
We find that pre-trained English language models predict empirical moral norms across countries worse than the English moral norms reported previously.
- Score: 3.475552182166427
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Moral norms vary across cultures. A recent line of work suggests that English
large language models contain human-like moral biases, but these studies
typically do not examine moral variation in a diverse cultural setting. We
investigate the extent to which monolingual English language models contain
knowledge about moral norms in different countries. We consider two levels of
analysis: 1) whether language models capture fine-grained moral variation
across countries over a variety of topics such as ``homosexuality'' and
``divorce''; 2) whether language models capture cultural diversity and shared
tendencies in which topics people around the globe tend to diverge or agree on
in their moral judgment. We perform our analyses with two public datasets from
the World Values Survey (across 55 countries) and PEW global surveys (across 40
countries) on morality. We find that pre-trained English language models
predict empirical moral norms across countries worse than the English moral
norms reported previously. However, fine-tuning language models on the survey
data improves inference across countries at the expense of a less accurate
estimate of the English moral norms. We discuss the relevance and challenges of
incorporating cultural knowledge into the automated inference of moral norms.
Related papers
- Evaluating Moral Beliefs across LLMs through a Pluralistic Framework [22.0799438612003]
This study introduces a novel three-module framework to evaluate the moral beliefs of four prominent large language models.
We constructed a dataset containing 472 moral choice scenarios in Chinese, derived from moral words.
By ranking these moral choices, we discern the varying moral beliefs held by different language models.
arXiv Detail & Related papers (2024-11-06T04:52:38Z) - Language Model Alignment in Multilingual Trolley Problems [138.5684081822807]
Building on the Moral Machine experiment, we develop a cross-lingual corpus of moral dilemma vignettes in over 100 languages called MultiTP.
Our analysis explores the alignment of 19 different LLMs with human judgments, capturing preferences across six moral dimensions.
We discover significant variance in alignment across languages, challenging the assumption of uniform moral reasoning in AI systems.
arXiv Detail & Related papers (2024-07-02T14:02:53Z) - Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in
Large Language Models [89.94270049334479]
This paper identifies a cultural dominance issue within large language models (LLMs)
LLMs often provide inappropriate English-culture-related answers that are not relevant to the expected culture when users ask in non-English languages.
arXiv Detail & Related papers (2023-10-19T05:38:23Z) - Speaking Multiple Languages Affects the Moral Bias of Language Models [70.94372902010232]
Pre-trained multilingual language models (PMLMs) are commonly used when dealing with data from multiple languages and cross-lingual transfer.
Do the models capture moral norms from English and impose them on other languages?
Our experiments demonstrate that, indeed, PMLMs encode differing moral biases, but these do not necessarily correspond to cultural differences or commonalities in human opinions.
arXiv Detail & Related papers (2022-11-14T20:08:54Z) - NormSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations
On-the-Fly [61.77957329364812]
We introduce a framework for addressing the novel task of conversation-grounded multi-lingual, multi-cultural norm discovery.
NormSAGE elicits knowledge about norms through directed questions representing the norm discovery task and conversation context.
It further addresses the risk of language model hallucination with a self-verification mechanism ensuring that the norms discovered are correct.
arXiv Detail & Related papers (2022-10-16T18:30:05Z) - Do Multilingual Language Models Capture Differing Moral Norms? [71.52261949766101]
Massively multilingual sentence representations are trained on large corpora of uncurated data.
This may cause the models to grasp cultural values including moral judgments from the high-resource languages.
The lack of data in certain languages can also lead to developing random and thus potentially harmful beliefs.
arXiv Detail & Related papers (2022-03-18T12:26:37Z) - Identifying Morality Frames in Political Tweets using Relational
Learning [27.047907641503762]
Moral sentiment is motivated by its targets, which can correspond to individuals or collective entities.
We introduce morality frames, a representation framework for organizing moral attitudes directed at different entities.
We propose a relational learning model to predict moral attitudes towards entities and moral foundations jointly.
arXiv Detail & Related papers (2021-09-09T19:48:57Z) - Morality-based Assertion and Homophily on Social Media: A Cultural
Comparison between English and Japanese Languages [8.22469542459168]
We used the social media platform Twitter for comparing the moral behaviors of Japanese users with English users.
The tweets from Japanese users depicted relatively higher Fairness, Ingroup and Purity.
As far as emotions related to morality are concerned, the English tweets expressed more positive emotions for all moral dimensions.
arXiv Detail & Related papers (2021-08-24T11:07:46Z) - Aligning AI With Shared Human Values [85.2824609130584]
We introduce the ETHICS dataset, a new benchmark that spans concepts in justice, well-being, duties, virtues, and commonsense morality.
We find that current language models have a promising but incomplete ability to predict basic human ethical judgements.
Our work shows that progress can be made on machine ethics today, and it provides a steppingstone toward AI that is aligned with human values.
arXiv Detail & Related papers (2020-08-05T17:59:16Z) - Text-based inference of moral sentiment change [11.188112005462536]
We present a text-based framework for investigating moral sentiment change of the public via longitudinal corpora.
We build our methodology by exploring moral biases learned from diachronic word embeddings.
Our work offers opportunities for applying natural language processing toward characterizing moral sentiment change in society.
arXiv Detail & Related papers (2020-01-20T18:52:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.