Unsupervised Parallel Corpus Mining on Web Data

• URL: http://arxiv.org/abs/2009.08595v1
• Date: Fri, 18 Sep 2020 02:38:01 GMT
• Title: Unsupervised Parallel Corpus Mining on Web Data
• Authors: Guokun Lai, Zihang Dai, Yiming Yang
• Abstract summary: We present a pipeline to mine the parallel corpus from the Internet in an unsupervised manner. Our system produces new state-of-the-art results, 39.81 and 38.95 BLEU scores, even compared with supervised approaches.
• Score: 53.74427402568838
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With a large amount of parallel data, neural machine translation systems are able to deliver human-level performance for sentence-level translation. However, it is costly to label a large amount of parallel data by humans. In contrast, there is a large-scale of parallel corpus created by humans on the Internet. The major difficulty to utilize them is how to filter them out from the noise website environments. Current parallel data mining methods all require labeled parallel data as the training source. In this paper, we present a pipeline to mine the parallel corpus from the Internet in an unsupervised manner. On the widely used WMT'14 English-French and WMT'16 English-German benchmarks, the machine translator trained with the data extracted by our pipeline achieves very close performance to the supervised results. On the WMT'16 English-Romanian and Romanian-English benchmarks, our system produces new state-of-the-art results, 39.81 and 38.95 BLEU scores, even compared with supervised approaches.

Related papers

• Bilingual Corpus Mining and Multistage Fine-Tuning for Improving Machine Translation of Lecture Transcripts [50.00305136008848]
  We propose a framework for parallel corpus mining, which provides a quick and effective way to mine a parallel corpus from publicly available lectures on Coursera. For both English--Japanese and English--Chinese lecture translations, we extracted parallel corpora of approximately 50,000 lines and created development and test sets. This study also suggests guidelines for gathering and cleaning corpora, mining parallel sentences, cleaning noise in the mined data, and creating high-quality evaluation splits.
  arXiv  Detail & Related papers  (2023-11-07T03:50:25Z)
• Exploring Paracrawl for Document-level Neural Machine Translation [21.923881766940088]
  Document-level neural machine translation (NMT) has outperformed sentence-level NMT on a number of datasets. We show that document-level NMT models trained with only parallel paragraphs from Paracrawl can be used to translate real documents.
  arXiv  Detail & Related papers  (2023-04-20T11:21:34Z)
• PARTIME: Scalable and Parallel Processing Over Time with Deep Neural Networks [68.96484488899901]
  We present PARTIME, a library designed to speed up neural networks whenever data is continuously streamed over time. PARTIME starts processing each data sample at the time in which it becomes available from the stream. Experiments are performed in order to empirically compare PARTIME with classic non-parallel neural computations in online learning.
  arXiv  Detail & Related papers  (2022-10-17T14:49:14Z)
• Exploring Unsupervised Pretraining Objectives for Machine Translation [99.5441395624651]
  Unsupervised cross-lingual pretraining has achieved strong results in neural machine translation (NMT) Most approaches adapt masked-language modeling (MLM) to sequence-to-sequence architectures, by masking parts of the input and reconstructing them in the decoder. We compare masking with alternative objectives that produce inputs resembling real (full) sentences, by reordering and replacing words based on their context.
  arXiv  Detail & Related papers  (2021-06-10T10:18:23Z)
• Meta Back-translation [111.87397401837286]
  We propose a novel method to generate pseudo-parallel data from a pre-trained back-translation model. Our method is a meta-learning algorithm which adapts a pre-trained back-translation model so that the pseudo-parallel data it generates would train a forward-translation model to do well on a validation set.
  arXiv  Detail & Related papers  (2021-02-15T20:58:32Z)
• A Corpus for English-Japanese Multimodal Neural Machine Translation with Comparable Sentences [21.43163704217968]
  We propose a new multimodal English-Japanese corpus with comparable sentences that are compiled from existing image captioning datasets. Due to low translation scores in our baseline experiments, we believe that current multimodal NMT models are not designed to effectively utilize comparable sentence data.
  arXiv  Detail & Related papers  (2020-10-17T06:12:25Z)
• Unsupervised Bitext Mining and Translation via Self-trained Contextual Embeddings [51.47607125262885]
  We describe an unsupervised method to create pseudo-parallel corpora for machine translation (MT) from unaligned text. We use multilingual BERT to create source and target sentence embeddings for nearest-neighbor search and adapt the model via self-training. We validate our technique by extracting parallel sentence pairs on the BUCC 2017 bitext mining task and observe up to a 24.5 point increase (absolute) in F1 scores over previous unsupervised methods.
  arXiv  Detail & Related papers  (2020-10-15T14:04:03Z)
• Parallel Corpus Filtering via Pre-trained Language Models [14.689457985200141]
  Web-crawled data provides a good source of parallel corpora for training machine translation models. Recent work shows that neural machine translation systems are more sensitive to noise than traditional statistical machine translation methods. We propose a novel approach to filter out noisy sentence pairs from web-crawled corpora via pre-trained language models.
  arXiv  Detail & Related papers  (2020-05-13T06:06:23Z)
• Coursera Corpus Mining and Multistage Fine-Tuning for Improving Lectures Translation [37.04364877980479]
  We show how to mine a parallel corpus from publicly available lectures at Coursera. Our approach determines sentence alignments, relying on machine translation and cosine similarity over continuous-space sentence representations. For Japanese--English lectures translation, we extracted parallel data of approximately 40,000 lines and created development and test sets.
  arXiv  Detail & Related papers  (2019-12-26T01:12:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.