Knowledge of cultural moral norms in large language models
- URL: http://arxiv.org/abs/2306.01857v1
- Date: Fri, 2 Jun 2023 18:23:35 GMT
- Title: Knowledge of cultural moral norms in large language models
- Authors: Aida Ramezani, Yang Xu
- Abstract summary: We investigate the extent to which monolingual English language models contain knowledge about moral norms in different countries.
We perform our analyses with two public datasets from the World Values Survey and PEW global surveys on morality.
We find that pre-trained English language models predict empirical moral norms across countries worse than the English moral norms reported previously.
- Score: 3.475552182166427
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Moral norms vary across cultures. A recent line of work suggests that English
large language models contain human-like moral biases, but these studies
typically do not examine moral variation in a diverse cultural setting. We
investigate the extent to which monolingual English language models contain
knowledge about moral norms in different countries. We consider two levels of
analysis: 1) whether language models capture fine-grained moral variation
across countries over a variety of topics such as ``homosexuality'' and
``divorce''; 2) whether language models capture cultural diversity and shared
tendencies in which topics people around the globe tend to diverge or agree on
in their moral judgment. We perform our analyses with two public datasets from
the World Values Survey (across 55 countries) and PEW global surveys (across 40
countries) on morality. We find that pre-trained English language models
predict empirical moral norms across countries worse than the English moral
norms reported previously. However, fine-tuning language models on the survey
data improves inference across countries at the expense of a less accurate
estimate of the English moral norms. We discuss the relevance and challenges of
incorporating cultural knowledge into the automated inference of moral norms.
Related papers
- Multilingual Trolley Problems for Language Models [138.0995992619116]
This study is inspired by a large-scale cross-cultural study of human moral preferences, "The Moral Machine Experiment"
We show that large language models (LLMs) are more aligned with human preferences in languages such as English, Korean, Hungarian, and Chinese, but less aligned in languages such as Hindi and Somali (in Africa)
We also characterize the explanations LLMs give for their moral choices and find that fairness is the most dominant supporting reason behind GPT-4's decisions and utilitarianism by GPT-3.
arXiv Detail & Related papers (2024-07-02T14:02:53Z) - MoralBERT: A Fine-Tuned Language Model for Capturing Moral Values in Social Discussions [4.747987317906765]
Moral values play a fundamental role in how we evaluate information, make decisions, and form judgements around important social issues.
Recent advances in Natural Language Processing (NLP) show that moral values can be gauged in human-generated textual content.
This paper introduces MoralBERT, a range of language representation models fine-tuned to capture moral sentiment in social discourse.
arXiv Detail & Related papers (2024-03-12T14:12:59Z) - Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in
Large Language Models [89.94270049334479]
This paper identifies a cultural dominance issue within large language models (LLMs)
LLMs often provide inappropriate English-culture-related answers that are not relevant to the expected culture when users ask in non-English languages.
arXiv Detail & Related papers (2023-10-19T05:38:23Z) - Speaking Multiple Languages Affects the Moral Bias of Language Models [70.94372902010232]
Pre-trained multilingual language models (PMLMs) are commonly used when dealing with data from multiple languages and cross-lingual transfer.
Do the models capture moral norms from English and impose them on other languages?
Our experiments demonstrate that, indeed, PMLMs encode differing moral biases, but these do not necessarily correspond to cultural differences or commonalities in human opinions.
arXiv Detail & Related papers (2022-11-14T20:08:54Z) - NormSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations
On-the-Fly [61.77957329364812]
We introduce a framework for addressing the novel task of conversation-grounded multi-lingual, multi-cultural norm discovery.
NormSAGE elicits knowledge about norms through directed questions representing the norm discovery task and conversation context.
It further addresses the risk of language model hallucination with a self-verification mechanism ensuring that the norms discovered are correct.
arXiv Detail & Related papers (2022-10-16T18:30:05Z) - Do Multilingual Language Models Capture Differing Moral Norms? [71.52261949766101]
Massively multilingual sentence representations are trained on large corpora of uncurated data.
This may cause the models to grasp cultural values including moral judgments from the high-resource languages.
The lack of data in certain languages can also lead to developing random and thus potentially harmful beliefs.
arXiv Detail & Related papers (2022-03-18T12:26:37Z) - Identifying Morality Frames in Political Tweets using Relational
Learning [27.047907641503762]
Moral sentiment is motivated by its targets, which can correspond to individuals or collective entities.
We introduce morality frames, a representation framework for organizing moral attitudes directed at different entities.
We propose a relational learning model to predict moral attitudes towards entities and moral foundations jointly.
arXiv Detail & Related papers (2021-09-09T19:48:57Z) - Morality-based Assertion and Homophily on Social Media: A Cultural
Comparison between English and Japanese Languages [8.22469542459168]
We used the social media platform Twitter for comparing the moral behaviors of Japanese users with English users.
The tweets from Japanese users depicted relatively higher Fairness, Ingroup and Purity.
As far as emotions related to morality are concerned, the English tweets expressed more positive emotions for all moral dimensions.
arXiv Detail & Related papers (2021-08-24T11:07:46Z) - Aligning AI With Shared Human Values [85.2824609130584]
We introduce the ETHICS dataset, a new benchmark that spans concepts in justice, well-being, duties, virtues, and commonsense morality.
We find that current language models have a promising but incomplete ability to predict basic human ethical judgements.
Our work shows that progress can be made on machine ethics today, and it provides a steppingstone toward AI that is aligned with human values.
arXiv Detail & Related papers (2020-08-05T17:59:16Z) - Text-based inference of moral sentiment change [11.188112005462536]
We present a text-based framework for investigating moral sentiment change of the public via longitudinal corpora.
We build our methodology by exploring moral biases learned from diachronic word embeddings.
Our work offers opportunities for applying natural language processing toward characterizing moral sentiment change in society.
arXiv Detail & Related papers (2020-01-20T18:52:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.