Towards Cross-lingual Values Assessment: A Consensus-Pluralism Perspective

• URL: http://arxiv.org/abs/2602.17283v1
• Date: Thu, 19 Feb 2026 11:41:34 GMT
• Title: Towards Cross-lingual Values Assessment: A Consensus-Pluralism Perspective
• Authors: Yukun Chen, Xinyu Zhang, Jialong Tang, Yu Wan, Baosong Yang, Yiming Li, Zhan Qin, Kui Ren,
• Abstract summary: X-Value is a novel Cross-lingual Values Assessment Benchmark designed to assess deep-level values of content from a global perspective.<n>X-Value consists of more than 5,000 QA pairs across 18 languages, systematically organized into 7 core domains grounded in Schwartz's Theory of Basic Human Values.
• Score: 47.001227248208856
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While large language models (LLMs) have become pivotal to content safety, current evaluation paradigms primarily focus on detecting explicit harms (e.g., violence or hate speech), neglecting the subtler value dimensions conveyed in digital content. To bridge this gap, we introduce X-Value, a novel Cross-lingual Values Assessment Benchmark designed to evaluate LLMs' ability to assess deep-level values of content from a global perspective. X-Value consists of more than 5,000 QA pairs across 18 languages, systematically organized into 7 core domains grounded in Schwartz's Theory of Basic Human Values and categorized into easy and hard levels for discriminative evaluation. We further propose a unique two-stage annotation framework that first identifies whether an issue falls under global consensus (e.g., human rights) or pluralism (e.g., religion), and subsequently conducts a multi-party evaluation of the latent values embedded within the content. Systematic evaluations on X-Value reveal that current SOTA LLMs exhibit deficiencies in cross-lingual values assessment ($Acc < 77\%$), with significant performance disparities across different languages ($ΔAcc > 20\%$). This work highlights the urgent need to improve the nuanced, values-aware content assessment capability of LLMs. Our X-Value is available at: https://huggingface.co/datasets/Whitolf/X-Value.

Related papers

• Evaluating Large Language Model with Knowledge Oriented Language Specific Simple Question Answering [73.73820209993515]
  We introduce KoLasSimpleQA, the first benchmark evaluating the multilingual factual ability of Large Language Models (LLMs)<n>Inspired by existing research, we created the question set with features such as single knowledge point coverage, absolute objectivity, unique answers, and temporal stability.<n>Results show significant performance differences between the two domains.
  arXiv  Detail & Related papers  (2025-05-22T12:27:02Z)
• Value Portrait: Assessing Language Models' Values through Psychometrically and Ecologically Valid Items [2.9357382494347264]
  Existing benchmarks rely on human or machine annotations that are vulnerable to value-related biases.<n>We propose the Value Portrait benchmark, which consists of items that capture real-life user-LLM interactions.<n>This psychometrically validated approach ensures that items strongly correlated with specific values serve as reliable items for assessing those values.
  arXiv  Detail & Related papers  (2025-05-02T05:26:50Z)
• Value Compass Benchmarks: A Platform for Fundamental and Validated Evaluation of LLMs Values [76.70893269183684]
  Large Language Models (LLMs) achieve remarkable breakthroughs.<n> aligning their values with humans has become imperative for their responsible development.<n>There still lack evaluations of LLMs values that fulfill three desirable goals.
  arXiv  Detail & Related papers  (2025-01-13T05:53:56Z)
• MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models [71.36392373876505]
  We introduce MMIE, a large-scale benchmark for evaluating interleaved multimodal comprehension and generation in Large Vision-Language Models (LVLMs)<n>MMIE comprises 20K meticulously curated multimodal queries, spanning 3 categories, 12 fields, and 102 subfields, including mathematics, coding, physics, literature, health, and arts.<n>It supports both interleaved inputs and outputs, offering a mix of multiple-choice and open-ended question formats to evaluate diverse competencies.
  arXiv  Detail & Related papers  (2024-10-14T04:15:00Z)
• CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responses [34.77031649891843]
  We introduce CLAVE, a novel framework which integrates two complementary Large Language Models (LLMs) This dual-model approach enables calibration with any value systems using 100 human-labeled samples per value type. We present ValEval, a comprehensive dataset comprising 13k+ (text,value,label) 12+s across diverse domains, covering three major value systems.
  arXiv  Detail & Related papers  (2024-07-15T13:51:37Z)
• Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Values [47.779186412943076]
  We propose a novel basic value alignment paradigm and a value space spanned by basic value dimensions. Inspired by basic values in humanity and social science across cultures, this work proposes a novel basic value alignment paradigm and a value space spanned by basic value dimensions. To foster future research, we apply the representative Schwartz's Theory of Basic Values as an example and construct FULCRA, a dataset consisting of 5k (LLM output, value vector) pairs.
  arXiv  Detail & Related papers  (2023-11-15T10:29:28Z)
• CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibility [62.74405775089802]
  We present CValues, the first Chinese human values evaluation benchmark to measure the alignment ability of LLMs. As a result, we have manually collected adversarial safety prompts across 10 scenarios and induced responsibility prompts from 8 domains. Our findings suggest that while most Chinese LLMs perform well in terms of safety, there is considerable room for improvement in terms of responsibility.
  arXiv  Detail & Related papers  (2023-07-19T01:22:40Z)
• Heterogeneous Value Alignment Evaluation for Large Language Models [91.96728871418]
  Large Language Models (LLMs) have made it crucial to align their values with those of humans. We propose a Heterogeneous Value Alignment Evaluation (HVAE) system to assess the success of aligning LLMs with heterogeneous values.
  arXiv  Detail & Related papers  (2023-05-26T02:34:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.