Multilingual Fine-Tuning via Localized Gradient Conflict Resolution

• URL: http://arxiv.org/abs/2606.05613v1
• Date: Thu, 04 Jun 2026 02:36:30 GMT
• Title: Multilingual Fine-Tuning via Localized Gradient Conflict Resolution
• Abstract summary: We introduce Bucket-Level MOO, a scalable distributed framework that applies gradient-based MOO algorithms locally.<n> Empirically, Bucket-Level MOOs mitigate interference by driving Large Language Models to construct distinct language-specific dimensions.<n>Our method significantly improves both seen and unseen multilingual performance over standard fine-tuning paradigms.
• Score: 20.41455787951831
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid evolution of Large Language Models (LLMs) has established cross-lingual versatility as a defining feature of modern systems. However, fine-tuning these models frequently induces negative interference across languages. To address this, we reformulate multilingual fine-tuning as a multi-objective optimization (MOO) problem. Specifically, we introduce Bucket-Level MOO, a scalable distributed framework that applies gradient-based MOO algorithms locally on parameter buckets. This enables conflict-aware updates without the prohibitive communication overhead of reconstructing full gradient vectors. Theoretically, we prove this localized resolution natively enforces Refined Pareto Stationarity, a strictly tighter necessary condition for Pareto optimality. Empirically, Bucket-Level MOO mitigates interference by driving LLMs to construct distinct language-specific dimensions, improving representational separability. Extensive experiments across four base LLMs demonstrate that our method significantly improves both seen and unseen multilingual performance over standard fine-tuning paradigms.

Related papers

• TLPO: Token-Level Policy Optimization for Mitigating Language Confusion in Large Language Models [6.468917070386353]
  Large language models (LLMs) demonstrate strong multilingual capabilities, yet often fail to consistently generate responses in the intended language.<n>We introduce Token-Level Policy Optimization (TLPO), a fine-tuning framework designed to mitigate language confusion through localized, token-level updates.
  arXiv  Detail & Related papers  (2026-04-29T11:39:43Z)
• COMPASS: COntinual Multilingual PEFT with Adaptive Semantic Sampling [0.023074632109535153]
  We introduce a data-centric framework for adapting large language models to target languages.<n>We use a distribution-aware sampling strategy to identify semantic gaps between existing training data and a target usage distribution.<n>We extend this into a continual learning framework, which monitors for data distribution shifts in production and dynamically updates adapters to prevent model staleness.
  arXiv  Detail & Related papers  (2026-04-22T16:07:10Z)
• LVRPO: Language-Visual Alignment with GRPO for Multimodal Understanding and Generation [51.071351994330605]
  Unified multimodal pretraining has emerged as a promising paradigm for jointly modeling language and vision within a single foundation model.<n>Existing approaches largely rely on implicit or indirect alignment signals and remain suboptimal for simultaneously supporting multimodal understanding and generation.<n>We propose LVRPO, a language-visual reinforcement-based preference optimization framework that explicitly aligns language and visual representations.
  arXiv  Detail & Related papers  (2026-03-29T13:38:21Z)
• Multi-Paradigm Collaborative Adversarial Attack Against Multi-Modal Large Language Models [67.45032003041399]
  We propose a novel Multi-Paradigm Collaborative Attack (MPCAttack) framework to boost the transferability of adversarial examples against MLLMs.<n>MPCO adaptively balances the importance of different paradigm representations and guides the global optimisation.<n>Our solution consistently outperforms state-of-the-art methods in both targeted and untargeted attacks on open-source and closed-source MLLMs.
  arXiv  Detail & Related papers  (2026-03-05T06:01:26Z)
• Optimizing Language Models for Crosslingual Knowledge Consistency [90.86445137816942]
  Large language models are known to often exhibit inconsistent knowledge.<n>This is particularly problematic in multilingual scenarios, where models are likely to be asked similar questions in different languages.<n>In this work, we show that this issue can be mitigated using reinforcement learning with a structured reward function.
  arXiv  Detail & Related papers  (2026-03-04T23:36:55Z)
• Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignment [15.241143079313757]
  We introduce a plug-and-play Multi-Lingual Consistency (MLC) loss that can be integrated into existing monolingual alignment pipelines.<n>This allows simultaneous alignment across multiple languages using only multilingual prompt variants without requiring additional semantic response-level supervision in low-resource languages.
  arXiv  Detail & Related papers  (2026-02-18T18:01:23Z)
• Multi-Objective Hierarchical Optimization with Large Language Models [41.41567058185742]
  Large Language Models (LLMs) are not the off-the-shelf choice to drive multi-objective optimization yet.<n>In this paper, we close this gap by leveraging LLMs as surrogate models and candidate samplers inside a structured hierarchical search strategy.
  arXiv  Detail & Related papers  (2026-01-20T12:10:13Z)
• PlaM: Training-Free Plateau-Guided Model Merging for Better Visual Grounding in MLLMs [59.78917775399492]
  Multimodal instruction fine-tuning paradoxically degrades this text's reasoning capability.<n>We propose a training-free framework to mitigate this degradation.
  arXiv  Detail & Related papers  (2026-01-12T15:27:51Z)
• Topological Alignment of Shared Vision-Language Embedding Space [5.5522557994489246]
  ToMCLIP is a topology-aware framework aligning embedding spaces with topology-preserving constraints.<n>We show enhanced structural coherence of multilingual representations, higher zero-shot accuracy on the CIFAR-100, and stronger multilingual retrieval performance on the xFlickr&CO.
  arXiv  Detail & Related papers  (2025-10-13T01:36:38Z)
• The Unreasonable Effectiveness of Model Merging for Cross-Lingual Transfer in LLMs [45.08958917457921]
  Large language models (LLMs) still struggle across tasks outside of high-resource languages.<n>In this work, we investigate cross-lingual transfer to lower-resource languages where task-specific post-training data is scarce.
  arXiv  Detail & Related papers  (2025-05-23T20:28:31Z)
• Multi-Reference Preference Optimization for Large Language Models [56.84730239046117]
  We introduce a novel closed-form formulation for direct preference optimization using multiple reference models. The resulting algorithm, Multi-Reference Preference Optimization (MRPO), leverages broader prior knowledge from diverse reference models. Our experiments demonstrate that LLMs finetuned with MRPO generalize better in various preference data, regardless of data scarcity or abundance.
  arXiv  Detail & Related papers  (2024-05-26T00:29:04Z)
• Distributionally Robust Multilingual Machine Translation [94.51866646879337]
  We propose a new learning objective for Multilingual neural machine translation (MNMT) based on distributionally robust optimization. We show how to practically optimize this objective for large translation corpora using an iterated best response scheme. Our method consistently outperforms strong baseline methods in terms of average and per-language performance under both many-to-one and one-to-many translation settings.
  arXiv  Detail & Related papers  (2021-09-09T03:48:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.