Steering LLMs for Culturally Localized Generation

• URL: http://arxiv.org/abs/2603.23301v1
• Date: Tue, 24 Mar 2026 15:04:01 GMT
• Title: Steering LLMs for Culturally Localized Generation
• Abstract summary: We identify interpretable features that encode culturally salient information and aggregate them into Cultural Embeddings (CuE)<n>CuE-based steering increases cultural faithfulness and elicits significantly rarer, long-tail cultural concepts than prompting alone.<n>This also suggests that models do benefit from better elicitation strategies, and don't necessarily lack long-tail knowledge representation.
• Score: 29.19138401210989
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLMs are deployed globally, yet produce responses biased towards cultures with abundant training data. Existing cultural localization approaches such as prompting or post-training alignment are black-box, hard to control, and do not reveal whether failures reflect missing knowledge or poor elicitation. In this paper, we address these gaps using mechanistic interpretability to uncover and manipulate cultural representations in LLMs. Leveraging sparse autoencoders, we identify interpretable features that encode culturally salient information and aggregate them into Cultural Embeddings (CuE). We use CuE both to analyze implicit cultural biases under underspecified prompts and to construct white-box steering interventions. Across multiple models, we show that CuE-based steering increases cultural faithfulness and elicits significantly rarer, long-tail cultural concepts than prompting alone. Notably, CuE-based steering is complementary to black-box localization methods, offering gains when applied on top of prompt-augmented inputs. This also suggests that models do benefit from better elicitation strategies, and don't necessarily lack long-tail knowledge representation, though this varies across cultures. Our results provide both diagnostic insight into cultural representations in LLMs and a controllable method to steer towards desired cultures.

Related papers

• LLMs Infer Cultural Context but Fail to Apply It When Responding [17.31205603254047]
  We introduce Cultural and Pragmatic Response Inference (CAPRI), a dataset of conversations with varying levels of cultural cues.<n> Experiments with state-of-the-art LLMs show that models can infer cultural background and recall relevant conventions, but often fail to utilize the information.<n>We find that models increasingly adapt their answers as cultural cues accumulate, but their priors are not culture-neutral, sometimes aligning with the model's country of origin.
  arXiv  Detail & Related papers  (2026-06-16T08:53:24Z)
• CultureForest: Understanding and Evaluating Cultural Norm Grounded Reasoning in LLMs [59.09971492475217]
  We introduce CultureForest, a benchmark for textitCultural Norm Grounded Reasoning<n>CultureForest comprises 5,378 examples across 8 domains and 53 countries/regions.<n>Extensive experiments reveal that even top-tier models degrade substantially in open-ended settings.
  arXiv  Detail & Related papers  (2026-06-01T08:25:12Z)
• CrossCult-KIBench: A Benchmark for Cross-Cultural Knowledge Insertion in MLLMs [13.267566050457974]
  Cross-cultural knowledge insertion focuses on adapting models to specific cultural contexts while preserving original behavior in other cultures.<n> benchmark includes 9,800 image-grounded cases covering 49 culturally relevant visual scenarios across English, Chinese, and Arabic language-culture groups.<n>Memory-Conditioned Knowledge Insertion (MCKI) retrieves relevant cultural knowledge from an external memory using frozen MLLM representations.
  arXiv  Detail & Related papers  (2026-05-07T12:26:11Z)
• LLMs as Cultural Archives: Cultural Commonsense Knowledge Graph Extraction [57.23766971626989]
  Large language models (LLMs) encode rich cultural knowledge learned from diverse web-scale data.<n>We present an iterative, prompt-based framework for constructing a Cultural Commonsense Knowledge Graph (CCKG)<n>We find that the cultural knowledge graphs are better realized in English, even when the target culture is non-English.
  arXiv  Detail & Related papers  (2026-01-25T20:05:04Z)
• CultureScope: A Dimensional Lens for Probing Cultural Understanding in LLMs [57.653830744706305]
  CultureScope is the most comprehensive evaluation framework to date for assessing cultural understanding in large language models.<n>Inspired by the cultural iceberg theory, we design a novel dimensional schema for cultural knowledge classification.<n> Experimental results demonstrate that our method can effectively evaluate cultural understanding.
  arXiv  Detail & Related papers  (2025-09-19T17:47:48Z)
• Entangled in Representations: Mechanistic Investigation of Cultural Biases in Large Language Models [42.367959511140036]
  We propose Culturescope, the first interpretability-based method that probes the internal representations of large language models.<n>We introduce a cultural flattening score as a measure of the intrinsic cultural biases.<n>Our experimental results reveal that LLMs encode Western-dominance bias and cultural flattening in their cultural knowledge space.
  arXiv  Detail & Related papers  (2025-08-12T12:05:32Z)
• From Surveys to Narratives: Rethinking Cultural Value Adaptation in LLMs [62.9861554207279]
  Adapting cultural values in Large Language Models (LLMs) presents significant challenges.<n>Prior work primarily aligns LLMs with different cultural values using World Values Survey (WVS) data.<n>We investigate WVS-based training for cultural value adaptation and find that relying solely on survey data cane cultural norms and interfere with factual knowledge.
  arXiv  Detail & Related papers  (2025-05-22T09:00:01Z)
• Through the Prism of Culture: Evaluating LLMs' Understanding of Indian Subcultures and Traditions [9.331687165284587]
  We evaluate the capacity of Large Language Models to recognize and accurately respond to the Little Traditions within Indian society.<n>Through a series of case studies, we assess whether LLMs can balance the interplay between dominant Great Traditions and localized Little Traditions.<n>Our findings reveal that while LLMs demonstrate an ability to articulate cultural nuances, they often struggle to apply this understanding in practical, context-specific scenarios.
  arXiv  Detail & Related papers  (2025-01-28T06:58:25Z)
• Self-Pluralising Culture Alignment for Large Language Models [36.689491885394034]
  We propose CultureSPA, a framework that allows large language models to align to pluralistic cultures. By comparing culture-aware/unaware outputs, we are able to detect and collect culture-related instances. Extensive experiments demonstrate that CultureSPA significantly improves the alignment of LLMs to diverse cultures without compromising general abilities.
  arXiv  Detail & Related papers  (2024-10-16T19:06:08Z)
• Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsense [98.09670425244462]
  Large language models (LLMs) have demonstrated substantial commonsense understanding. This paper examines the capabilities and limitations of several state-of-the-art LLMs in the context of cultural commonsense tasks.
  arXiv  Detail & Related papers  (2024-05-07T20:28:34Z)
• Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models [89.94270049334479]
  This paper identifies a cultural dominance issue within large language models (LLMs) LLMs often provide inappropriate English-culture-related answers that are not relevant to the expected culture when users ask in non-English languages.
  arXiv  Detail & Related papers  (2023-10-19T05:38:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.