CLEAR: Cross-Lingual Enhancement in Alignment via Reverse-training

• URL: http://arxiv.org/abs/2604.05821v2
• Date: Tue, 14 Apr 2026 08:05:53 GMT
• Title: CLEAR: Cross-Lingual Enhancement in Alignment via Reverse-training
• Abstract summary: Cross-Lingual Enhancement in Retrieval via Reverse-training (CLEAR) is a novel loss function utilizing a reverse training scheme to improve retrieval performance.<n>CLEAR achieves notable improvements in cross-lingual scenarios, with gains up to 15%, particularly in low-resource languages.
• Score: 25.752819016822574
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing multilingual embedding models often encounter challenges in cross-lingual scenarios due to imbalanced linguistic resources and less consideration of cross-lingual alignment during training. Although standardized contrastive learning approaches for cross-lingual adaptation are widely adopted, they may struggle to capture fundamental alignment between languages and degrade performance in well-aligned languages such as English. To address these challenges, we propose Cross-Lingual Enhancement in Retrieval via Reverse-training (CLEAR), a novel loss function utilizing a reverse training scheme to improve retrieval performance across diverse cross-lingual retrieval scenarios. CLEAR leverages an English passage as a bridge to strengthen alignments between the target language and English, ensuring robust performance in the cross-lingual retrieval task. Our extensive experiments demonstrate that CLEAR achieves notable improvements in cross-lingual scenarios, with gains up to 15%, particularly in low-resource languages, while minimizing performance degradation in English. Furthermore, our findings highlight that CLEAR offers promising effectiveness even in multilingual training, suggesting its potential for broad application and scalability. We release the code at https://github.com/dltmddbs100/CLEAR.

Related papers

• IRIS: Interleaved Reinforcement with Incremental Staged Curriculum for Cross-Lingual Mathematical Reasoning [54.28485292978118]
  IRIS is a two-axis framework that combinesSupervised Fine-Tuning on progressively harder problems with Reverse Curriculum Reinforcement Learning.<n>We release CL-Math, a dataset of 29k problems with step-level annotations in English, Hindi, and Marathi.<n>IRIS consistently improves performance across standard benchmarks and curated multilingual test sets.
  arXiv  Detail & Related papers  (2026-04-27T07:10:35Z)
• Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance [30.9640918372872]
  Large Language Models (LLMs) struggle with cross-lingual tasks due to data imbalances between high-resource and low-resource languages.<n>Existing methods, such as bilingual fine-tuning and contrastive alignment, can improve cross-lingual performance.<n>We introduce a Cross-Lingual Mapping Task during the pre-training phase, which enhances cross-lingual alignment without compromising monolingual fluency.
  arXiv  Detail & Related papers  (2026-04-12T11:43:25Z)
• Code-Switching In-Context Learning for Cross-Lingual Transfer of Large Language Models [64.54005959758733]
  We introduce code-switching in-context learning (CSICL) as a principled and robust approach for overcoming the translation barrier during inference.<n>We conduct extensive experiments across 4 LLMs, 6 datasets, and 10 languages, spanning both knowledge-intensive and reasoning-oriented domains.<n>Our results demonstrate CSICL consistently outperforms X-ICL baselines, achieving gains of 3.1%p and 1.9%p in both target and unseen languages.
  arXiv  Detail & Related papers  (2025-10-07T08:35:42Z)
• Cross-Lingual Pitfalls: Automatic Probing Cross-Lingual Weakness of Multilingual Large Language Models [55.14276067678253]
  This paper introduces a novel methodology for efficiently identifying inherent cross-lingual weaknesses in Large Language Models (LLMs)<n>We construct a new dataset of over 6,000 bilingual pairs across 16 languages using this methodology, demonstrating its effectiveness in revealing weaknesses even in state-of-the-art models.<n>Further experiments investigate the relationship between linguistic similarity and cross-lingual weaknesses, revealing that linguistically related languages share similar performance patterns.
  arXiv  Detail & Related papers  (2025-05-24T12:31:27Z)
• Align, Generate, Learn: A Novel Closed-Loop Framework for Cross-Lingual In-Context Learning [0.0]
  Cross-lingual in-context learning (XICL) has emerged as a transformative paradigm for leveraging large language models (LLMs) to tackle multilingual tasks.<n>We propose a novel self-supervised framework that harnesses the generative capabilities of LLMs to internally select and utilize task-relevant examples.
  arXiv  Detail & Related papers  (2024-12-12T05:36:51Z)
• Lens: Rethinking Multilingual Enhancement for Large Language Models [70.85065197789639]
  We propose Lens, a novel approach to enhance multilingual capabilities in large language models (LLMs)<n>Lens operates on two subspaces: the language-agnostic subspace, where it aligns target languages with the central language to inherit strong semantic representations, and the language-specific subspace, where it separates target and central languages to preserve linguistic specificity.<n>Lens significantly improves multilingual performance while maintaining the model's English proficiency, achieving better results with less computational cost compared to existing post-training approaches.
  arXiv  Detail & Related papers  (2024-10-06T08:51:30Z)
• No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement [59.37775534633868]
  We introduce a novel method called language arithmetic, which enables training-free post-processing. The effectiveness of the proposed solution is demonstrated on three downstream tasks in a MAD-X-based set of cross-lingual schemes.
  arXiv  Detail & Related papers  (2024-04-24T08:52:40Z)
• CrossIn: An Efficient Instruction Tuning Approach for Cross-Lingual Knowledge Alignment [38.35458193262633]
  English-centric models are usually suboptimal in other languages.<n>We propose a novel approach called CrossIn, which utilizes a mixed composition of cross-lingual instruction tuning data.
  arXiv  Detail & Related papers  (2024-04-18T06:20:50Z)
• From Classification to Generation: Insights into Crosslingual Retrieval Augmented ICL [8.065775937617417]
  We introduce a novel approach that leverages cross-lingual retrieval-augmented in-context learning (CREA-ICL) By extracting semantically similar prompts from high-resource languages, we aim to improve the zero-shot performance of multilingual pre-trained language models (MPLMs) Though our approach yields steady improvements in classification tasks, it faces challenges in generation tasks.
  arXiv  Detail & Related papers  (2023-11-11T15:40:21Z)
• RC3: Regularized Contrastive Cross-lingual Cross-modal Pre-training [84.23022072347821]
  We propose a regularized cross-lingual visio-textual contrastive learning objective that constrains the representation proximity of weakly-aligned visio-textual inputs. Experiments on 5 downstream multi-modal tasks across 6 languages demonstrate the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2023-05-13T14:41:05Z)
• A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning [6.329304732560936]
  Existing zero-shot cross-lingual transfer methods rely on parallel corpora or bilingual dictionaries. We propose Embedding-Push, Attention-Pull, and Robust targets to transfer English embeddings to virtual multilingual embeddings without semantic loss.
  arXiv  Detail & Related papers  (2022-10-18T15:36:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.