Rethinking Cross-lingual Alignment: Balancing Transfer and Cultural Erasure in Multilingual LLMs

• URL: http://arxiv.org/abs/2510.26024v1
• Date: Wed, 29 Oct 2025 23:37:54 GMT
• Title: Rethinking Cross-lingual Alignment: Balancing Transfer and Cultural Erasure in Multilingual LLMs
• Abstract summary: Cross-lingual alignment aims to align multilingual representations, enabling Large Language Models to seamlessly transfer knowledge across languages.<n>While intuitive, we hypothesize, this pursuit of representational convergence can inadvertently cause "cultural erasure"<n>We propose Surgical Steering, a novel inference-time method that disentangles these two objectives.
• Score: 13.444040667645979
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cross-lingual alignment (CLA) aims to align multilingual representations, enabling Large Language Models (LLMs) to seamlessly transfer knowledge across languages. While intuitive, we hypothesize, this pursuit of representational convergence can inadvertently cause "cultural erasure", the functional loss of providing culturally-situated responses that should diverge based on the query language. In this work, we systematically analyze this trade-off by introducing a holistic evaluation framework, the transfer-localization plane, which quantifies both desirable knowledge transfer and undesirable cultural erasure. Using this framework, we re-evaluate recent CLA approaches and find that they consistently improve factual transfer at the direct cost of cultural localization across all six languages studied. Our investigation into the internal representations of these models reveals a key insight: universal factual transfer and culturally-specific knowledge are optimally steerable at different model layers. Based on this finding, we propose Surgical Steering, a novel inference-time method that disentangles these two objectives. By applying targeted activation steering to distinct layers, our approach achieves a better balance between the two competing dimensions, effectively overcoming the limitations of current alignment techniques.

Related papers

• Cross-Lingual Consensus: Aligning Multilingual Cultural Knowledge via Multilingual Self-Consistency [1.338174941551702]
  Large Language Models (LLMs) exhibit significant performance discrepancies across languages.<n>We propose a novel self-supervised framework to bridge this knowledge gap.
  arXiv  Detail & Related papers  (2026-05-21T08:11:01Z)
• When Meanings Meet: Investigating the Emergence and Quality of Shared Concept Spaces during Multilingual Language Model Training [57.230355403478995]
  We investigate the development of language-agnostic concept spaces during pretraining of EuroLLM.<n>We find that shared concept spaces emerge early and continue to refine, but that alignment with them is language-dependent.<n>In contrast to prior work, our fine-grained manual analysis reveals that some apparent gains in translation quality reflect shifts in behavior.
  arXiv  Detail & Related papers  (2026-01-30T11:23:01Z)
• CLM-Bench: Benchmarking and Analyzing Cross-lingual Misalignment of LLMs in Knowledge Editing [5.137059606366328]
  We propose CLM-Bench, a culture-aware benchmark constructed using a native Chinese-first methodology.<n>We conduct extensive experiments on representative LLMs and reveal a significant Cross-lingual Misalignment.<n>Our findings challenge the effectiveness of current methods in cross-lingual transfer and underscore the importance of culturally native benchmarks.
  arXiv  Detail & Related papers  (2026-01-24T09:55:34Z)
• On the Entity-Level Alignment in Crosslingual Consistency [62.33186691736433]
  SubSub and SubInj integrate English translations of subjects into prompts across languages, leading to substantial gains in factual recall accuracy and consistency.<n>These interventions reinforce the entity representation alignment in the conceptual space through model's internal pivot-language processing.
  arXiv  Detail & Related papers  (2025-10-11T16:26:50Z)
• MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation [91.22008265721952]
  MMA-ASIA centers on a human-curated, multilingual, and multimodally aligned benchmark covering 8 Asian countries and 10 languages.<n>This is the first dataset aligned at the input level across three modalities: text, image (visual question answering), and speech.<n>We propose a five-dimensional evaluation protocol that measures: (i) cultural-awareness disparities across countries, (ii) cross-lingual consistency, (iii) cross-modal consistency, (iv) cultural knowledge generalization, and (v) grounding validity.
  arXiv  Detail & Related papers  (2025-10-07T14:12:12Z)
• Cross-Lingual Transfer of Cultural Knowledge: An Asymmetric Phenomenon [31.04530022789729]
  We study how cultural knowledge transfers across languages during language adaptation of large language models (LLMs)<n>We observe bidirectional cultural transfer between English and other high-resource languages, while low-resource languages primarily transfer knowledge to English with limited reverse flow.<n>To explain this asymmetric phenomenon, we propose a frequency-based hypothesis: cultural knowledge appearing more frequently in the pretraining data transfers more easily.
  arXiv  Detail & Related papers  (2025-06-02T13:45:09Z)
• Disentangling Language and Culture for Evaluating Multilingual Large Language Models [48.06219053598005]
  This paper introduces a Dual Evaluation Framework to comprehensively assess the multilingual capabilities of LLMs.<n>By decomposing the evaluation along the dimensions of linguistic medium and cultural context, this framework enables a nuanced analysis of LLMs' ability to process questions cross-lingually.
  arXiv  Detail & Related papers  (2025-05-30T14:25:45Z)
• From Word to World: Evaluate and Mitigate Culture Bias in LLMs via Word Association Test [50.51344198689069]
  We extend the human-centered word association test (WAT) to assess the alignment of large language models with cross-cultural cognition.<n>To address culture preference, we propose CultureSteer, an innovative approach by embedding cultural-specific semantic associations directly within the model's internal representation space.
  arXiv  Detail & Related papers  (2025-05-24T07:05:10Z)
• High-Dimensional Interlingual Representations of Large Language Models [65.77317753001954]
  Large language models (LLMs) trained on massive multilingual datasets hint at the formation of interlingual constructs.<n>We explore 31 diverse languages varying on their resource-levels, typologies, and geographical regions.<n>We find that multilingual LLMs exhibit inconsistent cross-lingual alignments.
  arXiv  Detail & Related papers  (2025-03-14T10:39:27Z)
• Exploring Cross-lingual Latent Transplantation: Mutual Opportunities and Open Challenges [48.96952594416528]
  Current large language models (LLMs) often exhibit imbalances in multilingual capabilities and cultural adaptability.<n>XTransplant framework enables models to harness the complementary strengths of both English and non-English resources by transplanting latent activations across languages.
  arXiv  Detail & Related papers  (2024-12-17T09:05:30Z)
• Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalization [27.368684663279463]
  We investigate the potential for explicitly aligning conceptual correspondence between languages to enhance cross-lingual generalization. Using the syntactic aspect of language as a testbed, our analyses of 43 languages reveal a high degree of alignability. We propose a meta-learning-based method to learn to align conceptual spaces of different languages.
  arXiv  Detail & Related papers  (2023-10-19T14:50:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.