CultureLLM: Incorporating Cultural Differences into Large Language
Models
- URL: http://arxiv.org/abs/2402.10946v1
- Date: Fri, 9 Feb 2024 04:02:43 GMT
- Title: CultureLLM: Incorporating Cultural Differences into Large Language
Models
- Authors: Cheng Li, Mengzhou Chen, Jindong Wang, Sunayana Sitaram, Xing Xie
- Abstract summary: CultureLLM is a cost-effective solution to incorporate cultural differences into large language models.
Our human study shows that the generated samples are semantically equivalent to the original samples.
- Score: 39.33251733412784
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Large language models (LLMs) are reported to be partial to certain cultures
owing to the training data dominance from the English corpora. Since
multilingual cultural data are often expensive to collect, existing efforts
handle this by prompt engineering or culture-specific pre-training. However,
they might overlook the knowledge deficiency of low-resource culture and
require extensive computing resources. In this paper, we propose CultureLLM, a
cost-effective solution to incorporate cultural differences into LLMs.
CultureLLM adopts World Value Survey (WVS) as seed data and generates
semantically equivalent training data via the proposed semantic data
augmentation. Using only 50 seed samples from WVS with augmented data, we
fine-tune culture-specific LLMs and one unified model (CultureLLM-One) for 9
cultures covering rich and low-resource languages. Extensive experiments on 60
culture-related datasets demonstrate that CultureLLM significantly outperforms
various counterparts such as GPT-3.5 (by 8.1%) and Gemini Pro (by 9.5%) with
comparable performance to GPT-4 or even better. Our human study shows that the
generated samples are semantically equivalent to the original samples,
providing an effective solution for LLMs augmentation.
Related papers
- Crossroads of Continents: Automated Artifact Extraction for Cultural Adaptation with Large Multimodal Models [22.92083941222383]
We first introduce Dalle Street, a large-scale dataset containing 9,935 images of 67 countries and 10 concept classes.
Next, we assess models' deeper culture understanding by an artifact extraction task and identify over 18,000 artifacts associated with different countries.
Finally, we propose a highly composable pipeline, CultureAdapt, to adapt images from culture to culture.
arXiv Detail & Related papers (2024-07-02T08:55:41Z) - CulturePark: Boosting Cross-cultural Understanding in Large Language Models [63.452948673344395]
This paper introduces CulturePark, an LLM-powered multi-agent communication framework for cultural data collection.
It generates high-quality cross-cultural dialogues encapsulating human beliefs, norms, and customs.
We evaluate these models across three downstream tasks: content moderation, cultural alignment, and cultural education.
arXiv Detail & Related papers (2024-05-24T01:49:02Z) - CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Prompting [68.37589899302161]
We uncover culture perceptions of three SOTA models on 110 countries and regions on 8 culture-related topics through culture-conditioned generations.
We discover that culture-conditioned generation consist of linguistic "markers" that distinguish marginalized cultures apart from default cultures.
arXiv Detail & Related papers (2024-04-16T00:50:43Z) - Massively Multi-Cultural Knowledge Acquisition & LM Benchmarking [48.21982147529661]
This paper introduces a novel approach for massively multicultural knowledge acquisition.
Our method strategically navigates from densely informative Wikipedia documents on cultural topics to an extensive network of linked pages.
Our work marks an important step towards deeper understanding and bridging the gaps of cultural disparities in AI.
arXiv Detail & Related papers (2024-02-14T18:16:54Z) - CDEval: A Benchmark for Measuring the Cultural Dimensions of Large Language Models [41.885600036131045]
CDEval is a benchmark aimed at evaluating the cultural dimensions of Large Language Models.
It is constructed by incorporating both GPT-4's automated generation and human verification, covering six cultural dimensions across seven domains.
arXiv Detail & Related papers (2023-11-28T02:01:25Z) - Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in
Large Language Models [89.94270049334479]
This paper identifies a cultural dominance issue within large language models (LLMs)
LLMs often provide inappropriate English-culture-related answers that are not relevant to the expected culture when users ask in non-English languages.
arXiv Detail & Related papers (2023-10-19T05:38:23Z) - Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensions [10.415002561977655]
This research proposes a Cultural Alignment Test (Hoftede's CAT) to quantify cultural alignment using Hofstede's cultural dimension framework.
We quantitatively evaluate large language models (LLMs) against the cultural dimensions of regions like the United States, China, and Arab countries.
Our results quantify the cultural alignment of LLMs and reveal the difference between LLMs in explanatory cultural dimensions.
arXiv Detail & Related papers (2023-08-25T14:50:13Z) - Benchmarking LLM-based Machine Translation on Cultural Awareness [53.83912076814508]
Translating cultural-specific content is crucial for effective cross-cultural communication.
Recent advancements in in-context learning utilize lightweight prompts to guide large language models (LLMs) in machine translation tasks.
We introduce a new data curation pipeline to construct a culturally relevant parallel corpus.
arXiv Detail & Related papers (2023-05-23T17:56:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.