Gradient-Informed Training for Low-Resource Multilingual Speech Translation

• URL: http://arxiv.org/abs/2603.25836v1
• Date: Thu, 26 Mar 2026 19:00:00 GMT
• Title: Gradient-Informed Training for Low-Resource Multilingual Speech Translation
• Abstract summary: This work proposes a principled methodology to automatically determine layer-specific sharing patterns by mining training gradient information.<n>Our approach employs three distinct analysis strategies: distance-based language clustering, self/cross-task divergence metrics for capacity allocation, and joint factorization.
• Score: 4.344398827902069
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In low-resource multilingual speech-to-text translation, uniform architectural sharing across languages frequently introduces representation conflicts that impede convergence. This work proposes a principled methodology to automatically determine layer-specific sharing patterns by mining training gradient information. Our approach employs three distinct analysis strategies: distance-based language clustering, self/cross-task divergence metrics for capacity allocation, and joint factorization coupled with canonical correlation analysis for subspace alignment. Extensive evaluation across four language pairs (using the SeamlessM4T-Medium architecture) demonstrates persistent improvements in translation quality metrics.

Related papers

• Disentangled Contrastive Learning for Zero-Shot Multilingual Dense Retrieval [18.12664959335536]
  We propose a disentangled contrastive learning(DCL) method for multilingual dense retrieval.<n>Specifically, we design disentangled optimization objectives based on hierarchical semantic alignment and language debiasing contrastive learning.<n>Our method consistently outperforms several strong baselines, demonstrating its effectiveness and generalization ability.
  arXiv  Detail & Related papers  (2026-08-03T13:13:15Z)
• Multilinguality as Sense Adaptation [24.548610248136352]
  SENse-based Symmetric Interlingual Alignment (SENSIA)<n>We introduce SENse-based Symmetric Interlingual Alignment (SENSIA)<n>It adapts a Backpack language model from one language to another by explicitly aligning sense-level mixtures and contextual representations on parallel data.
  arXiv  Detail & Related papers  (2026-01-15T11:44:01Z)
• Enhancing LLM Language Adaption through Cross-lingual In-Context Pre-training [57.62126373849383]
  Cross-lingual In-context Pre-training (CrossIC-PT) is a simple and scalable approach that enhances cross-lingual transfer.<n>We construct CrossIC-PT samples by interleaving semantic-related bilingual Wikipedia documents into a single context window.<n> Experimental results demonstrate that CrossIC-PT improves multilingual performance on three models across six target languages.
  arXiv  Detail & Related papers  (2025-04-29T07:24:25Z)
• Optimal Transport Posterior Alignment for Cross-lingual Semantic Parsing [68.47787275021567]
  Cross-lingual semantic parsing transfers parsing capability from a high-resource language (e.g., English) to low-resource languages with scarce training data. We propose a new approach to cross-lingual semantic parsing by explicitly minimizing cross-lingual divergence between latent variables using Optimal Transport.
  arXiv  Detail & Related papers  (2023-07-09T04:52:31Z)
• Cross-Align: Modeling Deep Cross-lingual Interactions for Word Alignment [63.0407314271459]
  The proposed Cross-Align achieves the state-of-the-art (SOTA) performance on four out of five language pairs. Experiments show that the proposed Cross-Align achieves the state-of-the-art (SOTA) performance on four out of five language pairs.
  arXiv  Detail & Related papers  (2022-10-09T02:24:35Z)
• Multilingual Mix: Example Interpolation Improves Multilingual Neural Machine Translation [45.77509642452541]
  We introduce multilingual crossover encoder-decoder (mXEncDec) to fuse language pairs at an instance level. Our approach interpolates instances from different language pairs into joint crossover examples' in order to encourage sharing input and output spaces across languages.
  arXiv  Detail & Related papers  (2022-03-15T03:56:22Z)
• Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Training [58.72619374790418]
  MultiUAT dynamically adjusts the training data usage based on the model's uncertainty. We analyze the cross-domain transfer and show the deficiency of static and similarity based methods.
  arXiv  Detail & Related papers  (2021-09-06T08:30:33Z)
• Cross-lingual Text Classification with Heterogeneous Graph Neural Network [2.6936806968297913]
  Cross-lingual text classification aims at training a classifier on the source language and transferring the knowledge to target languages. Recent multilingual pretrained language models (mPLM) achieve impressive results in cross-lingual classification tasks. We propose a simple yet effective method to incorporate heterogeneous information within and across languages for cross-lingual text classification.
  arXiv  Detail & Related papers  (2021-05-24T12:45:42Z)
• Unsupervised Word Translation Pairing using Refinement based Point Set Registration [8.568050813210823]
  Cross-lingual alignment of word embeddings play an important role in knowledge transfer across languages. Current unsupervised approaches rely on similarities in geometric structure of word embedding spaces across languages. This paper proposes BioSpere, a novel framework for unsupervised mapping of bi-lingual word embeddings onto a shared vector space.
  arXiv  Detail & Related papers  (2020-11-26T09:51:29Z)
• Cross-lingual Spoken Language Understanding with Regularized Representation Alignment [71.53159402053392]
  We propose a regularization approach to align word-level and sentence-level representations across languages without any external resource. Experiments on the cross-lingual spoken language understanding task show that our model outperforms current state-of-the-art methods in both few-shot and zero-shot scenarios.
  arXiv  Detail & Related papers  (2020-09-30T08:56:53Z)
• Fine-Grained Analysis of Cross-Linguistic Syntactic Divergences [18.19093600136057]
  We propose a framework for extracting divergence patterns for any language pair from a parallel corpus. We show that our framework provides a detailed picture of cross-language divergences, generalizes previous approaches, and lends itself to full automation.
  arXiv  Detail & Related papers  (2020-05-07T13:05:03Z)
• Robust Cross-lingual Embeddings from Parallel Sentences [65.85468628136927]
  We propose a bilingual extension of the CBOW method which leverages sentence-aligned corpora to obtain robust cross-lingual word representations. Our approach significantly improves crosslingual sentence retrieval performance over all other approaches. It also achieves parity with a deep RNN method on a zero-shot cross-lingual document classification task.
  arXiv  Detail & Related papers  (2019-12-28T16:18:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.