Explicit Sentence Compression for Neural Machine Translation

• URL: http://arxiv.org/abs/1912.11980v1
• Date: Fri, 27 Dec 2019 04:14:06 GMT
• Title: Explicit Sentence Compression for Neural Machine Translation
• Authors: Zuchao Li, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita, Zhuosheng Zhang, Hai Zhao
• Abstract summary: State-of-the-art Transformer-based neural machine translation (NMT) systems still follow a standard encoder-decoder framework. backbone information, which stands for the gist of a sentence, is not specifically focused on. We propose an explicit sentence compression method to enhance the source sentence representation for NMT.
• Score: 110.98786673598016
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: State-of-the-art Transformer-based neural machine translation (NMT) systems still follow a standard encoder-decoder framework, in which source sentence representation can be well done by an encoder with self-attention mechanism. Though Transformer-based encoder may effectively capture general information in its resulting source sentence representation, the backbone information, which stands for the gist of a sentence, is not specifically focused on. In this paper, we propose an explicit sentence compression method to enhance the source sentence representation for NMT. In practice, an explicit sentence compression goal used to learn the backbone information in a sentence. We propose three ways, including backbone source-side fusion, target-side fusion, and both-side fusion, to integrate the compressed sentence into NMT. Our empirical tests on the WMT English-to-French and English-to-German translation tasks show that the proposed sentence compression method significantly improves the translation performances over strong baselines.

Related papers

• High-Quality Data Augmentation for Low-Resource NMT: Combining a Translation Memory, a GAN Generator, and Filtering [1.8843687952462742]
  This paper proposes a novel way of utilizing a monolingual corpus on the source side to assist Neural Machine Translation (NMT) in low-resource settings. We realize this concept by employing a Generative Adversarial Network (GAN), which augments the training data for the discriminator while mitigating the interference of low-quality synthetic monolingual translations with the generator.
  arXiv  Detail & Related papers  (2024-08-22T02:35:47Z)
• Categorizing Semantic Representations for Neural Machine Translation [53.88794787958174]
  We introduce categorization to the source contextualized representations. The main idea is to enhance generalization by reducing sparsity and overfitting. Experiments on a dedicated MT dataset show that our method reduces compositional generalization error rates by 24% error reduction.
  arXiv  Detail & Related papers  (2022-10-13T04:07:08Z)
• Principled Paraphrase Generation with Parallel Corpora [52.78059089341062]
  We formalize the implicit similarity function induced by round-trip Machine Translation. We show that it is susceptible to non-paraphrase pairs sharing a single ambiguous translation. We design an alternative similarity metric that mitigates this issue.
  arXiv  Detail & Related papers  (2022-05-24T17:22:42Z)
• Text Compression-aided Transformer Encoding [77.16960983003271]
  We propose explicit and implicit text compression approaches to enhance the Transformer encoding. backbone information, meaning the gist of the input text, is not specifically focused on. Our evaluation on benchmark datasets shows that the proposed explicit and implicit text compression approaches improve results in comparison to strong baselines.
  arXiv  Detail & Related papers  (2021-02-11T11:28:39Z)
• Code-switching pre-training for neural machine translation [13.35263905025371]
  This paper proposes a new pre-training method, called Code-Switching Pre-training (CSP) for Neural Machine Translation (NMT) Unlike traditional pre-training method which randomly masks some fragments of the input sentence, the proposed CSP randomly replaces some words in the source sentence with their translation words in the target language. Experimental results show that CSP achieves significant improvements over baselines without pre-training or with other pre-training methods.
  arXiv  Detail & Related papers  (2020-09-17T06:10:07Z)
• Explicit Reordering for Neural Machine Translation [50.70683739103066]
  In Transformer-based neural machine translation (NMT), the positional encoding mechanism helps the self-attention networks to learn the source representation with order dependency. We propose a novel reordering method to explicitly model this reordering information for the Transformer-based NMT. The empirical results on the WMT14 English-to-German, WAT ASPEC Japanese-to-English, and WMT17 Chinese-to-English translation tasks show the effectiveness of the proposed approach.
  arXiv  Detail & Related papers  (2020-04-08T05:28:46Z)
• Learning Contextualized Sentence Representations for Document-Level Neural Machine Translation [59.191079800436114]
  Document-level machine translation incorporates inter-sentential dependencies into the translation of a source sentence. We propose a new framework to model cross-sentence dependencies by training neural machine translation (NMT) to predict both the target translation and surrounding sentences of a source sentence.
  arXiv  Detail & Related papers  (2020-03-30T03:38:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.