CCBENCH: Assessing LLM Cultural Competence via Implicitly Signaled Norms using Health Queries

• URL: http://arxiv.org/abs/2607.05405v1
• Date: Mon, 08 Jun 2026 22:52:34 GMT
• Title: CCBENCH: Assessing LLM Cultural Competence via Implicitly Signaled Norms using Health Queries
• Abstract summary: We introduce CCBENCH, a framework for evaluating cultural competency in large language models (LLMs)<n>As a case study on health, we create CCBENCH-Health, which includes 60 theoretically grounded personas exhibiting varied norm-adherence states across six cultures.<n> Benchmarking five leading models reveals that even the best achieve culturally appropriate responses only 20-30% of the time.
• Score: 42.15271207948674
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To interact with users fairly and without stereotyping, AI models must display cultural competency, i.e., the ability to infer and adapt to a user's implicitly signaled cultural values, rather than relying on static demographic traits. We introduce CCBENCH, a framework for evaluating cultural competency in large language models (LLMs), treating culture as a continuum of norm adherence states rather than as a binary state of cultural belongingness. As a case study on health, we create CCBENCH-Health, which includes 60 theoretically grounded personas exhibiting varied norm-adherence states across six cultures, each engaging in 18 realistic dialogues. Each persona is evaluated on 52 authentic healthcare questions drawn from real user forums, yielding 3,120 unique interactions. Benchmarking five leading models reveals that even the best achieve culturally appropriate responses only 20-30% of the time. When explicitly prompted to focus on culturally relevant cues from the conversational history (CoT), performance improves modestly by 3-5% on average. We find that models perform best when personas avoid cultural norms rather than follow them, revealing a persistent asymmetry, suggesting a preference in the models to align with built-in biases than adapt to cultural cues. This is especially observed in the Afghan context (Avg: 8.8%), where cultural cues rarely yield appropriate health advice. Finally, we find that models sometimes adapt more readily to implicit, cultural conversational styles than to explicitly stated cultural practices, though this varies across cultures.

Related papers

• LLMs Infer Cultural Context but Fail to Apply It When Responding [17.31205603254047]
  We introduce Cultural and Pragmatic Response Inference (CAPRI), a dataset of conversations with varying levels of cultural cues.<n> Experiments with state-of-the-art LLMs show that models can infer cultural background and recall relevant conventions, but often fail to utilize the information.<n>We find that models increasingly adapt their answers as cultural cues accumulate, but their priors are not culture-neutral, sometimes aligning with the model's country of origin.
  arXiv  Detail & Related papers  (2026-06-16T08:53:24Z)
• Whose Norms? Disentangling Cultural and Personal Alignment in Large Language Models [66.34115263472931]
  Large language models are increasingly used for social decision-making situations that require balancing cultural norms with personal preferences.<n>We introduce PACT, the Personal-Preference and Cultural-Norm Trade-off framework, which evaluates whether models choose to follow a cultural norm or allow personal preferences.<n>Our five-country human study on PACT shows that culture-following in humans is mainly driven by scenario country, with the lowest agreement when participants judge their own cultural contexts.
  arXiv  Detail & Related papers  (2026-06-05T22:20:30Z)
• CultureForest: Understanding and Evaluating Cultural Norm Grounded Reasoning in LLMs [59.09971492475217]
  We introduce CultureForest, a benchmark for textitCultural Norm Grounded Reasoning<n>CultureForest comprises 5,378 examples across 8 domains and 53 countries/regions.<n>Extensive experiments reveal that even top-tier models degrade substantially in open-ended settings.
  arXiv  Detail & Related papers  (2026-06-01T08:25:12Z)
• The American Ghost in the Machine: How language models align culturally and the effects of cultural prompting [0.0]
  We use the VSM13 International Survey and Hofstede's cultural dimensions to identify the cultural alignment of popular Large Language Models (LLMs)<n>We then use cultural prompting to test the adaptability of these models to other cultures, namely China, France, India, Iran, Japan, and the United States.<n>We find that the majority of the eight LLMs tested favor the United States when the culture is not specified, with varying results when prompted for other cultures.
  arXiv  Detail & Related papers  (2025-12-13T23:11:41Z)
• Hire Your Anthropologist! Rethinking Culture Benchmarks Through an Anthropological Lens [9.000522371422628]
  We introduce a four-part framework that categorizes how benchmarks frame culture.<n>We qualitatively examine 20 cultural benchmarks and identify six recurring methodological issues.<n>Our aim is to guide the development of cultural benchmarks that go beyond static recall tasks.
  arXiv  Detail & Related papers  (2025-10-07T13:42:44Z)
• Cross-Cultural Transfer of Commonsense Reasoning in LLMs: Evidence from the Arab World [68.19795061447044]
  This paper investigates cross-cultural transfer of commonsense reasoning in the Arab world.<n>Using a culturally grounded commonsense reasoning dataset covering 13 Arab countries, we evaluate lightweight alignment methods.<n>Our results show that merely 12 culture-specific examples from one country can improve performance in others by 10% on average.
  arXiv  Detail & Related papers  (2025-09-23T17:24:14Z)
• CultureScope: A Dimensional Lens for Probing Cultural Understanding in LLMs [57.653830744706305]
  CultureScope is the most comprehensive evaluation framework to date for assessing cultural understanding in large language models.<n>Inspired by the cultural iceberg theory, we design a novel dimensional schema for cultural knowledge classification.<n> Experimental results demonstrate that our method can effectively evaluate cultural understanding.
  arXiv  Detail & Related papers  (2025-09-19T17:47:48Z)
• Fluent but Foreign: Even Regional LLMs Lack Cultural Alignment [24.871503011248777]
  Large language models (LLMs) are used worldwide, yet exhibit Western cultural tendencies.<n>We evaluate six Indic and six global LLMs on two dimensions -- values and practices.<n>Across tasks, Indic models do not align better with Indian norms than global models.
  arXiv  Detail & Related papers  (2025-05-25T01:59:23Z)
• CulturePark: Boosting Cross-cultural Understanding in Large Language Models [63.452948673344395]
  This paper introduces CulturePark, an LLM-powered multi-agent communication framework for cultural data collection. It generates high-quality cross-cultural dialogues encapsulating human beliefs, norms, and customs. We evaluate these models across three downstream tasks: content moderation, cultural alignment, and cultural education.
  arXiv  Detail & Related papers  (2024-05-24T01:49:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.