PAD: Towards Efficient Data Generation for Transfer Learning Using Phrase Alignment

• URL: http://arxiv.org/abs/2503.18250v2
• Date: Tue, 25 Mar 2025 23:16:28 GMT
• Title: PAD: Towards Efficient Data Generation for Transfer Learning Using Phrase Alignment
• Authors: Jong Myoung Kim, Young-Jun_Lee, Ho-Jin Choi, Sangkeun Jung,
• Abstract summary: We show that Phrase Aligned Data (PAD) synergizes effectively with the syntactic characteristics of the Korean language.<n>This innovative approach not only boosts model performance but also suggests a cost-efficient solution for resource-scarce languages.
• Score: 5.0560627648135785
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Transfer learning leverages the abundance of English data to address the scarcity of resources in modeling non-English languages, such as Korean. In this study, we explore the potential of Phrase Aligned Data (PAD) from standardized Statistical Machine Translation (SMT) to enhance the efficiency of transfer learning. Through extensive experiments, we demonstrate that PAD synergizes effectively with the syntactic characteristics of the Korean language, mitigating the weaknesses of SMT and significantly improving model performance. Moreover, we reveal that PAD complements traditional data construction methods and enhances their effectiveness when combined. This innovative approach not only boosts model performance but also suggests a cost-efficient solution for resource-scarce languages.

Related papers

• High-Resource Translation:Turning Abundance into Accessibility [0.0]
  This paper presents a novel approach to constructing an English-to-Telugu translation model by leveraging transfer learning techniques. The model incorporates iterative backtranslation to generate synthetic parallel data, effectively augmenting the training dataset and enhancing the model's translation capabilities.
  arXiv  Detail & Related papers  (2025-04-08T11:09:51Z)
• MoE-CT: A Novel Approach For Large Language Models Training With Resistance To Catastrophic Forgetting [53.77590764277568]
  We introduce a novel MoE-CT architecture that separates the base model's learning from the multilingual expansion process. Our design freezes the original LLM parameters, thus safeguarding its performance in high-resource languages, while an appended MoE module, trained on diverse language datasets, augments low-resource language proficiency.
  arXiv  Detail & Related papers  (2024-06-25T11:03:45Z)
• ALMol: Aligned Language-Molecule Translation LLMs through Offline Preference Contrastive Optimisation [2.296475290901356]
  We focus on machine language-molecule translation and deploy a novel training approach called contrastive preference optimisation. Our results demonstrate that our models achieve up to a 32% improvement compared to counterpart models.
  arXiv  Detail & Related papers  (2024-05-14T13:59:24Z)
• Improving Polish to English Neural Machine Translation with Transfer Learning: Effects of Data Volume and Language Similarity [2.4674086273775035]
  We investigate the impact of data volume and the use of similar languages on transfer learning in a machine translation task. We fine-tune mBART model for a Polish-English translation task using the OPUS-100 dataset. Our experiments show that a combination of related languages and larger amounts of data outperforms the model trained on related languages or larger amounts of data alone.
  arXiv  Detail & Related papers  (2023-06-01T13:34:21Z)
• Semi-supervised Neural Machine Translation with Consistency Regularization for Low-Resource Languages [3.475371300689165]
  This paper presents a simple yet effective method to tackle the problem for low-resource languages by augmenting high-quality sentence pairs and training NMT models in a semi-supervised manner. Specifically, our approach combines the cross-entropy loss for supervised learning with KL Divergence for unsupervised fashion given pseudo and augmented target sentences. Experimental results show that our approach significantly improves NMT baselines, especially on low-resource datasets with 0.46--2.03 BLEU scores.
  arXiv  Detail & Related papers  (2023-04-02T15:24:08Z)
• A Cohesive Distillation Architecture for Neural Language Models [0.0]
  A recent trend in Natural Language Processing is the exponential growth in Language Model (LM) size. This study investigates methods for Knowledge Distillation (KD) to provide efficient alternatives to large-scale models.
  arXiv  Detail & Related papers  (2023-01-12T08:01:53Z)
• Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translation [50.54059385277964]
  We present a novel data augmentation paradigm termed Continuous Semantic Augmentation (CsaNMT) CsaNMT augments each training instance with an adjacency region that could cover adequate variants of literal expression under the same meaning.
  arXiv  Detail & Related papers  (2022-04-14T08:16:28Z)
• Phrase-level Adversarial Example Generation for Neural Machine Translation [75.01476479100569]
  We propose a phrase-level adversarial example generation (PAEG) method to enhance the robustness of the model. We verify our method on three benchmarks, including LDC Chinese-English, IWSLT14 German-English, and WMT14 English-German tasks.
  arXiv  Detail & Related papers  (2022-01-06T11:00:49Z)
• On the Language Coverage Bias for Neural Machine Translation [81.81456880770762]
  Language coverage bias is important for neural machine translation (NMT) because the target-original training data is not well exploited in current practice. By carefully designing experiments, we provide comprehensive analyses of the language coverage bias in the training data. We propose two simple and effective approaches to alleviate the language coverage bias problem.
  arXiv  Detail & Related papers  (2021-06-07T01:55:34Z)
• Understanding and Improving Lexical Choice in Non-Autoregressive Translation [98.11249019844281]
  We propose to expose the raw data to NAT models to restore the useful information of low-frequency words. Our approach pushes the SOTA NAT performance on the WMT14 English-German and WMT16 Romanian-English datasets up to 27.8 and 33.8 BLEU points, respectively.
  arXiv  Detail & Related papers  (2020-12-29T03:18:50Z)
• Syntax-aware Data Augmentation for Neural Machine Translation [76.99198797021454]
  We propose a novel data augmentation strategy for neural machine translation. We set sentence-specific probability for word selection by considering their roles in sentence. Our proposed method is evaluated on WMT14 English-to-German dataset and IWSLT14 German-to-English dataset.
  arXiv  Detail & Related papers  (2020-04-29T13:45:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.