Beyond Human Norms: Unveiling Unique Values of Large Language Models through Interdisciplinary Approaches

• URL: http://arxiv.org/abs/2404.12744v2
• Date: Fri, 10 May 2024 06:09:02 GMT
• Title: Beyond Human Norms: Unveiling Unique Values of Large Language Models through Interdisciplinary Approaches
• Authors: Pablo Biedma, Xiaoyuan Yi, Linus Huang, Maosong Sun, Xing Xie,
• Abstract summary: This work proposes a novel framework, ValueLex, to reconstruct Large Language Models' unique value system from scratch. Based on Lexical Hypothesis, ValueLex introduces a generative approach to elicit diverse values from 30+ LLMs. We identify three core value dimensions, Competence, Character, and Integrity, each with specific subdimensions, revealing that LLMs possess a structured, albeit non-human, value system.
• Score: 69.73783026870998
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent advancements in Large Language Models (LLMs) have revolutionized the AI field but also pose potential safety and ethical risks. Deciphering LLMs' embedded values becomes crucial for assessing and mitigating their risks. Despite extensive investigation into LLMs' values, previous studies heavily rely on human-oriented value systems in social sciences. Then, a natural question arises: Do LLMs possess unique values beyond those of humans? Delving into it, this work proposes a novel framework, ValueLex, to reconstruct LLMs' unique value system from scratch, leveraging psychological methodologies from human personality/value research. Based on Lexical Hypothesis, ValueLex introduces a generative approach to elicit diverse values from 30+ LLMs, synthesizing a taxonomy that culminates in a comprehensive value framework via factor analysis and semantic clustering. We identify three core value dimensions, Competence, Character, and Integrity, each with specific subdimensions, revealing that LLMs possess a structured, albeit non-human, value system. Based on this system, we further develop tailored projective tests to evaluate and analyze the value inclinations of LLMs across different model sizes, training methods, and data sources. Our framework fosters an interdisciplinary paradigm of understanding LLMs, paving the way for future AI alignment and regulation.

Related papers

• Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Models [13.513813405118478]
  Large Language Models (LLMs) have raised concerns regarding their elusive intrinsic values. This study addresses the gap by introducing the Generative Psycho-Lexical Approach (GPLA) We propose a psychologically grounded five-factor value system tailored for LLMs.
  arXiv  Detail & Related papers  (2025-02-04T16:10:55Z)
• Value Compass Leaderboard: A Platform for Fundamental and Validated Evaluation of LLMs Values [76.70893269183684]
  Large Language Models (LLMs) achieve remarkable breakthroughs, aligning their values with humans has become imperative. Existing evaluations focus narrowly on safety risks such as bias and toxicity. Existing benchmarks are prone to data contamination. The pluralistic nature of human values across individuals and cultures is largely ignored in measuring LLMs value alignment.
  arXiv  Detail & Related papers  (2025-01-13T05:53:56Z)
• Quantifying Risk Propensities of Large Language Models: Ethical Focus and Bias Detection through Role-Play [0.43512163406552007]
  As Large Language Models (LLMs) become more prevalent, concerns about their safety, ethics, and potential biases have risen. This study innovatively applies the Domain-Specific Risk-Taking (DOSPERT) scale from cognitive science to LLMs. We propose a novel Ethical Decision-Making Risk Attitude Scale (EDRAS) to assess LLMs' ethical risk attitudes in depth.
  arXiv  Detail & Related papers  (2024-10-26T15:55:21Z)
• Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models [13.795641564238434]
  This work introduces Generative Psychometrics for Values (GPV) GPV is a data-driven value measurement paradigm grounded in text-revealed selective perceptions. Applying GPV to human-authored blogs, we demonstrate its stability, validity, and superiority over prior psychological tools.
  arXiv  Detail & Related papers  (2024-09-18T16:26:22Z)
• Unveiling the Misuse Potential of Base Large Language Models via In-Context Learning [61.2224355547598]
  Open-sourcing of large language models (LLMs) accelerates application development, innovation, and scientific progress. Our investigation exposes a critical oversight in this belief. By deploying carefully designed demonstrations, our research demonstrates that base LLMs could effectively interpret and execute malicious instructions.
  arXiv  Detail & Related papers  (2024-04-16T13:22:54Z)
• Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Values [47.779186412943076]
  We propose a novel basic value alignment paradigm and a value space spanned by basic value dimensions. Inspired by basic values in humanity and social science across cultures, this work proposes a novel basic value alignment paradigm and a value space spanned by basic value dimensions. To foster future research, we apply the representative Schwartz's Theory of Basic Values as an example and construct FULCRA, a dataset consisting of 5k (LLM output, value vector) pairs.
  arXiv  Detail & Related papers  (2023-11-15T10:29:28Z)
• A Survey on Evaluation of Large Language Models [87.60417393701331]
  Large language models (LLMs) are gaining increasing popularity in both academia and industry. This paper focuses on three key dimensions: what to evaluate, where to evaluate, and how to evaluate.
  arXiv  Detail & Related papers  (2023-07-06T16:28:35Z)
• Heterogeneous Value Alignment Evaluation for Large Language Models [91.96728871418]
  Large Language Models (LLMs) have made it crucial to align their values with those of humans. We propose a Heterogeneous Value Alignment Evaluation (HVAE) system to assess the success of aligning LLMs with heterogeneous values.
  arXiv  Detail & Related papers  (2023-05-26T02:34:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.