Sub-Word Alignment Is Still Useful: A Vest-Pocket Method for Enhancing Low-Resource Machine Translation

• URL: http://arxiv.org/abs/2205.04067v1
• Date: Mon, 9 May 2022 06:44:24 GMT
• Title: Sub-Word Alignment Is Still Useful: A Vest-Pocket Method for Enhancing Low-Resource Machine Translation
• Abstract summary: We leverage embedding duplication between aligned sub-words to extend the Parent-Child transfer learning method. We conduct experiments on benchmark datasets of My-En, Id-En and Tr-En translation scenarios.
• Score: 4.809907063232602
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We leverage embedding duplication between aligned sub-words to extend the Parent-Child transfer learning method, so as to improve low-resource machine translation. We conduct experiments on benchmark datasets of My-En, Id-En and Tr-En translation scenarios. The test results show that our method produces substantial improvements, achieving the BLEU scores of 22.5, 28.0 and 18.1 respectively. In addition, the method is computationally efficient which reduces the consumption of training time by 63.8%, reaching the duration of 1.6 hours when training on a Tesla 16GB P100 GPU. All the models and source codes in the experiments will be made publicly available to support reproducible research.

Related papers

• Text Quality-Based Pruning for Efficient Training of Language Models [66.66259229732121]
  We propose a novel method for numerically evaluating text quality in large unlabelled NLP datasets. By proposing the text quality metric, the paper establishes a framework to identify and eliminate low-quality text instances. Experimental results over multiple models and datasets demonstrate the efficacy of this approach.
  arXiv  Detail & Related papers  (2024-04-26T18:01:25Z)
• SelfSeg: A Self-supervised Sub-word Segmentation Method for Neural Machine Translation [51.881877192924414]
  Sub-word segmentation is an essential pre-processing step for Neural Machine Translation (NMT) This paper introduces SelfSeg, a self-supervised neural sub-word segmentation method. SelfSeg is much faster to train/decode and requires only monolingual dictionaries instead of parallel corpora.
  arXiv  Detail & Related papers  (2023-07-31T04:38:47Z)
• Strategies for improving low resource speech to text translation relying on pre-trained ASR models [59.90106959717875]
  This paper presents techniques and findings for improving the performance of low-resource speech to text translation (ST) We conducted experiments on both simulated and real-low resource setups, on language pairs English - Portuguese, and Tamasheq - French respectively.
  arXiv  Detail & Related papers  (2023-05-31T21:58:07Z)
• Efficient Fine-Tuning of Compressed Language Models with Learners [12.768368718187428]
  We introduce Learner modules and priming, novel methods for fine-tuning BERT-based models. Learner modules navigate the double bind of 1) training efficiently by fine-tuning a subset of parameters, and 2) training effectively by ensuring quick convergence and high metric scores. Our results on DistilBERT demonstrate that learners perform on par with or surpass the baselines.
  arXiv  Detail & Related papers  (2022-08-03T13:42:30Z)
• Improving Neural Machine Translation by Denoising Training [95.96569884410137]
  We present a simple and effective pretraining strategy Denoising Training DoT for neural machine translation. We update the model parameters with source- and target-side denoising tasks at the early stage and then tune the model normally. Experiments show DoT consistently improves the neural machine translation performance across 12 bilingual and 16 multilingual directions.
  arXiv  Detail & Related papers  (2022-01-19T00:11:38Z)
• Exploiting Curriculum Learning in Unsupervised Neural Machine Translation [28.75229367700697]
  We propose a curriculum learning method to gradually utilize pseudo bi-texts based on their quality from multiple granularities. Experimental results on WMT 14 En-Fr, WMT 16 En-De, WMT 16 En-Ro, and LDC En-Zh translation tasks demonstrate that the proposed method achieves consistent improvements with faster convergence speed.
  arXiv  Detail & Related papers  (2021-09-23T07:18:06Z)
• Efficient Nearest Neighbor Language Models [114.40866461741795]
  Non-parametric neural language models (NLMs) learn predictive distributions of text utilizing an external datastore. We show how to achieve up to a 6x speed-up in inference speed while retaining comparable performance.
  arXiv  Detail & Related papers  (2021-09-09T12:32:28Z)
• Cross-lingual Retrieval for Iterative Self-Supervised Training [66.3329263451598]
  Cross-lingual alignment can be further improved by training seq2seq models on sentence pairs mined using their own encoder outputs. We develop a new approach -- cross-lingual retrieval for iterative self-supervised training.
  arXiv  Detail & Related papers  (2020-06-16T21:30:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.