Measuring Spiritual Values and Bias of Large Language Models

• URL: http://arxiv.org/abs/2410.11647v1
• Date: Tue, 15 Oct 2024 14:33:23 GMT
• Title: Measuring Spiritual Values and Bias of Large Language Models
• Authors: Songyuan Liu, Ziyang Zhang, Runze Yan, Wei Wu, Carl Yang, Jiaying Lu,
• Abstract summary: Large language models (LLMs) have become integral tool for users from various backgrounds. These models reflect linguistic and cultural nuances embedded in pre-training data. values and perspectives inherent in this data can influence the behavior of LLMs, leading to potential biases.
• Score: 28.892254056685008
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) have become integral tool for users from various backgrounds. LLMs, trained on vast corpora, reflect the linguistic and cultural nuances embedded in their pre-training data. However, the values and perspectives inherent in this data can influence the behavior of LLMs, leading to potential biases. As a result, the use of LLMs in contexts involving spiritual or moral values necessitates careful consideration of these underlying biases. Our work starts with verification of our hypothesis by testing the spiritual values of popular LLMs. Experimental results show that LLMs' spiritual values are quite diverse, as opposed to the stereotype of atheists or secularists. We then investigate how different spiritual values affect LLMs in social-fairness scenarios e.g., hate speech identification). Our findings reveal that different spiritual values indeed lead to different sensitivity to different hate target groups. Furthermore, we propose to continue pre-training LLMs on spiritual texts, and empirical results demonstrate the effectiveness of this approach in mitigating spiritual bias.

Related papers

• An Evaluation of Cultural Value Alignment in LLM [27.437888319382893]
  We conduct the first large-scale evaluation of LLM culture assessing 20 countries' cultures and languages across ten LLMs. Our findings show that the output over all models represents a moderate cultural middle ground. Deeper investigation sheds light on the influence of model origin, prompt language, and value dimensions on cultural output.
  arXiv  Detail & Related papers  (2025-04-11T09:13:19Z)
• Value Compass Leaderboard: A Platform for Fundamental and Validated Evaluation of LLMs Values [76.70893269183684]
  Large Language Models (LLMs) achieve remarkable breakthroughs, aligning their values with humans has become imperative. Existing evaluations focus narrowly on safety risks such as bias and toxicity. Existing benchmarks are prone to data contamination. The pluralistic nature of human values across individuals and cultures is largely ignored in measuring LLMs value alignment.
  arXiv  Detail & Related papers  (2025-01-13T05:53:56Z)
• Large Language Models Reflect the Ideology of their Creators [73.25935570218375]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language. We uncover notable diversity in the ideological stance exhibited across different LLMs and languages.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• A Statistical Analysis of LLMs' Self-Evaluation Using Proverbs [1.9073729452914245]
  We introduce a novel proverb database consisting of 300 proverb pairs that are similar in intent but different in wording. We propose tests to evaluate textual consistencies as well as numerical consistencies across similar proverbs.
  arXiv  Detail & Related papers  (2024-10-22T02:38:48Z)
• Hate Personified: Investigating the role of LLMs in content moderation [64.26243779985393]
  For subjective tasks such as hate detection, where people perceive hate differently, the Large Language Model's (LLM) ability to represent diverse groups is unclear. By including additional context in prompts, we analyze LLM's sensitivity to geographical priming, persona attributes, and numerical information to assess how well the needs of various groups are reflected.
  arXiv  Detail & Related papers  (2024-10-03T16:43:17Z)
• Transforming Scholarly Landscapes: Influence of Large Language Models on Academic Fields beyond Computer Science [77.31665252336157]
  Large Language Models (LLMs) have ushered in a transformative era in Natural Language Processing (NLP) This work empirically examines the influence and use of LLMs in fields beyond NLP.
  arXiv  Detail & Related papers  (2024-09-29T01:32:35Z)
• Do Large Language Models Possess Sensitive to Sentiment? [18.88126980975737]
  Large Language Models (LLMs) have recently displayed their extraordinary capabilities in language understanding. This paper investigates the ability of LLMs to detect and react to sentiment in text modal.
  arXiv  Detail & Related papers  (2024-09-04T01:40:20Z)
• Exploring Value Biases: How LLMs Deviate Towards the Ideal [57.99044181599786]
  Large-Language-Models (LLMs) are deployed in a wide range of applications, and their response has an increasing social impact. We show that value bias is strong in LLMs across different categories, similar to the results found in human studies.
  arXiv  Detail & Related papers  (2024-02-16T18:28:43Z)
• Assessing LLMs for Moral Value Pluralism [2.860608352191896]
  We utilize a Recognizing Value Resonance (RVR) NLP model to identify World Values Survey (WVS) values that resonate and conflict with a given passage of text. We find that LLMs exhibit several Western-centric value biases. Our results highlight value misalignment and age groups, and a need for social science informed technological solutions.
  arXiv  Detail & Related papers  (2023-12-08T16:18:15Z)
• Are Large Language Models Really Robust to Word-Level Perturbations? [68.60618778027694]
  We propose a novel rational evaluation approach that leverages pre-trained reward models as diagnostic tools. Longer conversations manifest the comprehensive grasp of language models in terms of their proficiency in understanding questions. Our results demonstrate that LLMs frequently exhibit vulnerability to word-level perturbations that are commonplace in daily language usage.
  arXiv  Detail & Related papers  (2023-09-20T09:23:46Z)
• Quantifying the Impact of Large Language Models on Collective Opinion Dynamics [7.0012506428382375]
  We create an opinion network dynamics model to encode the opinions of large language models (LLMs) The results suggest that the output opinion of LLMs has a unique and positive effect on the collective opinion difference. Our experiments also find that introducing extra agents with opposite/neutral/random opinions, we can effectively mitigate the impact of biased/toxic output.
  arXiv  Detail & Related papers  (2023-08-07T05:45:17Z)
• A Survey on Evaluation of Large Language Models [87.60417393701331]
  Large language models (LLMs) are gaining increasing popularity in both academia and industry. This paper focuses on three key dimensions: what to evaluate, where to evaluate, and how to evaluate.
  arXiv  Detail & Related papers  (2023-07-06T16:28:35Z)
• Heterogeneous Value Alignment Evaluation for Large Language Models [91.96728871418]
  Large Language Models (LLMs) have made it crucial to align their values with those of humans. We propose a Heterogeneous Value Alignment Evaluation (HVAE) system to assess the success of aligning LLMs with heterogeneous values.
  arXiv  Detail & Related papers  (2023-05-26T02:34:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.