Improving Korean-English Cross-Lingual Retrieval: A Data-Centric Study of Language Composition and Model Merging

• URL: http://arxiv.org/abs/2507.08480v1
• Date: Fri, 11 Jul 2025 10:44:09 GMT
• Title: Improving Korean-English Cross-Lingual Retrieval: A Data-Centric Study of Language Composition and Model Merging
• Authors: Youngjoon Jang, Junyoung Son, Taemin Lee, Seongtae Hong, Heuiseok Lim,
• Abstract summary: We investigate the impact of training data composition on Cross-Lingual Information Retrieval ( CLIR) and Mono-Lingual Information Retrieval (IR) performance.<n>Our experiments reveal that the language composition of training data significantly influences IR performance, exhibiting important inter-lingual correlations.<n>Our work demonstrates that Model Merging can effectively mitigate this trade-off, achieving strong CLIR results while preserving Mono-Lingual IR capabilities.
• Score: 4.473623071673054
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the increasing utilization of multilingual text information, Cross-Lingual Information Retrieval (CLIR) has become a crucial research area. However, the impact of training data composition on both CLIR and Mono-Lingual Information Retrieval (IR) performance remains under-explored. To systematically investigate this data-centric aspect, we construct linguistically parallel Korean-English datasets and train retrieval models with various language combinations. Our experiments reveal that the language composition of training data significantly influences IR performance, exhibiting important inter-lingual correlations: CLIR performance improves with specific language pairs, while Mono-Lingual IR performance declines. Our work demonstrates that Model Merging can effectively mitigate this trade-off, achieving strong CLIR results while preserving Mono-Lingual IR capabilities. Our findings underscore the effects of linguistic configuration of training data on both CLIR and Mono-Lingual IR, and present Model Merging as a viable strategy to optimize performance across these tasks.

Related papers

• Language Mixing in Reasoning Language Models: Patterns, Impact, and Internal Causes [49.770097731093216]
  Reasoning language models (RLMs) excel at complex tasks by leveraging a chain-of-thought process to generate structured intermediate steps.<n> Language mixing, i.e., reasoning steps containing tokens from languages other than the prompt, has been observed in their outputs and shown to affect performance.<n>We present the first systematic study of language mixing in RLMs, examining its patterns, impact, and internal causes across 15 languages.
  arXiv  Detail & Related papers  (2025-05-20T18:26:53Z)
• Deciphering Cross-Modal Alignment in Large Vision-Language Models with Modality Integration Rate [118.37653302885607]
  We present the Modality Integration Rate (MIR), an effective, robust, and generalized metric to indicate the multi-modal pre-training quality of Large Vision Language Models (LVLMs) MIR is indicative about training data selection, training strategy schedule, and model architecture design to get better pre-training results.
  arXiv  Detail & Related papers  (2024-10-09T17:59:04Z)
• Multimodal Contrastive In-Context Learning [0.9120312014267044]
  This paper introduces a novel multimodal contrastive in-context learning framework to enhance our understanding of gradient-free in-context learning (ICL) in Large Language Models (LLMs) First, we present a contrastive learning-based interpretation of ICL in real-world settings, marking the distance of the key-value representation as the differentiator in ICL. Second, we develop an analytical framework to address biases in multimodal input formatting for real-world datasets. Third, we propose an on-the-fly approach for ICL that demonstrates effectiveness in detecting hateful memes.
  arXiv  Detail & Related papers  (2024-08-23T10:10:01Z)
• Evaluating Linguistic Capabilities of Multimodal LLMs in the Lens of Few-Shot Learning [15.919493497867567]
  This study aims to evaluate the performance of Multimodal Large Language Models (MLLMs) on the VALSE benchmark. We conducted a comprehensive assessment of state-of-the-art MLLMs, varying in model size and pretraining datasets.
  arXiv  Detail & Related papers  (2024-07-17T11:26:47Z)
• COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning [37.843051974342124]
  We introduce COIG-CQIA, a new Chinese instruction tuning dataset derived from various real-world resources and undergoing rigorous human verification. We conduct extensive experiments on COIG-CQIA, and compare them with strong baseline models and datasets. The experimental results show that models trained on COIG-CQIA achieve highly competitive performance in diverse benchmarks.
  arXiv  Detail & Related papers  (2024-03-26T19:24:18Z)
• Low-Rank Adaptation for Multilingual Summarization: An Empirical Study [60.541168233698194]
  We investigate the potential of. Efficient Fine-Tuning, focusing on Low-Rank Adaptation (LoRA) in the domain of multilingual summarization. We conduct an extensive study across different data availability scenarios, including high- and low-data settings, and cross-lingual transfer. Our findings reveal that LoRA is competitive with full fine-tuning when trained with high quantities of data, and excels in low-data scenarios and cross-lingual transfer.
  arXiv  Detail & Related papers  (2023-11-14T22:32:39Z)
• Dual-view Curricular Optimal Transport for Cross-lingual Cross-modal Retrieval [57.98555925471121]
  Cross-lingual cross-modal retrieval has attracted increasing attention. Most CCR methods construct pseudo-parallel vision-language corpora via Machine Translation. We propose Dual-view Curricular Optimal Transport (DCOT) to learn with noisy correspondence in CCR.
  arXiv  Detail & Related papers  (2023-09-11T13:44:46Z)
• Language Agnostic Multilingual Information Retrieval with Contrastive Learning [59.26316111760971]
  We present an effective method to train multilingual information retrieval systems. We leverage parallel and non-parallel corpora to improve the pretrained multilingual language models. Our model can work well even with a small number of parallel sentences.
  arXiv  Detail & Related papers  (2022-10-12T23:53:50Z)
• On Cross-Lingual Retrieval with Multilingual Text Encoders [51.60862829942932]
  We study the suitability of state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks. We benchmark their performance in unsupervised ad-hoc sentence- and document-level CLIR experiments. We evaluate multilingual encoders fine-tuned in a supervised fashion (i.e., we learn to rank) on English relevance data in a series of zero-shot language and domain transfer CLIR experiments.
  arXiv  Detail & Related papers  (2021-12-21T08:10:27Z)
• Mixed-Lingual Pre-training for Cross-lingual Summarization [54.4823498438831]
  Cross-lingual Summarization aims at producing a summary in the target language for an article in the source language. We propose a solution based on mixed-lingual pre-training that leverages both cross-lingual tasks like translation and monolingual tasks like masked language models. Our model achieves an improvement of 2.82 (English to Chinese) and 1.15 (Chinese to English) ROUGE-1 scores over state-of-the-art results.
  arXiv  Detail & Related papers  (2020-10-18T00:21:53Z)
• A Study of Neural Matching Models for Cross-lingual IR [17.89437720094451]
  We investigate interaction-based neural matching models for ad-hoc cross-lingual information retrieval ( CLIR) using cross-lingual word embeddings (CLWEs) With experiments conducted on the CLEF collection over four language pairs, we evaluate and provide insight into different neural model architectures. This study paves the way for learning an end-to-end CLIR system using CLWEs.
  arXiv  Detail & Related papers  (2020-05-26T19:21:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.