Principled Paraphrase Generation with Parallel Corpora
- URL: http://arxiv.org/abs/2205.12213v3
- Date: Tue, 23 May 2023 06:42:36 GMT
- Title: Principled Paraphrase Generation with Parallel Corpora
- Authors: Aitor Ormazabal, Mikel Artetxe, Aitor Soroa, Gorka Labaka and Eneko
Agirre
- Abstract summary: We formalize the implicit similarity function induced by round-trip Machine Translation.
We show that it is susceptible to non-paraphrase pairs sharing a single ambiguous translation.
We design an alternative similarity metric that mitigates this issue.
- Score: 52.78059089341062
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Round-trip Machine Translation (MT) is a popular choice for paraphrase
generation, which leverages readily available parallel corpora for supervision.
In this paper, we formalize the implicit similarity function induced by this
approach, and show that it is susceptible to non-paraphrase pairs sharing a
single ambiguous translation. Based on these insights, we design an alternative
similarity metric that mitigates this issue by requiring the entire translation
distribution to match, and implement a relaxation of it through the Information
Bottleneck method. Our approach incorporates an adversarial term into MT
training in order to learn representations that encode as much information
about the reference translation as possible, while keeping as little
information about the input as possible. Paraphrases can be generated by
decoding back to the source from this representation, without having to
generate pivot translations. In addition to being more principled and efficient
than round-trip MT, our approach offers an adjustable parameter to control the
fidelity-diversity trade-off, and obtains better results in our experiments.
Related papers
- Translating away Translationese without Parallel Data [14.423809260672877]
Translated texts exhibit systematic linguistic differences compared to original texts in the same language.
In this paper, we explore a novel approach to reduce translationese in translated texts: translation-based style transfer.
We show how we can eliminate the need for parallel validation data by combining the self-supervised loss with an unsupervised loss.
arXiv Detail & Related papers (2023-10-28T22:11:25Z) - Beyond Contrastive Learning: A Variational Generative Model for
Multilingual Retrieval [109.62363167257664]
We propose a generative model for learning multilingual text embeddings.
Our model operates on parallel data in $N$ languages.
We evaluate this method on a suite of tasks including semantic similarity, bitext mining, and cross-lingual question retrieval.
arXiv Detail & Related papers (2022-12-21T02:41:40Z) - STEMM: Self-learning with Speech-text Manifold Mixup for Speech
Translation [37.51435498386953]
We propose the Speech-TExt Manifold Mixup (STEMM) method to calibrate such discrepancy.
Experiments on MuST-C speech translation benchmark and further analysis show that our method effectively alleviates the cross-modal representation discrepancy.
arXiv Detail & Related papers (2022-03-20T01:49:53Z) - As Little as Possible, as Much as Necessary: Detecting Over- and
Undertranslations with Contrastive Conditioning [42.46681912294797]
We propose a method for detecting superfluous words in neural machine translation.
We compare the likelihood of a full sequence under a translation model to the likelihood of its parts, given the corresponding source or target sequence.
This allows to pinpoint superfluous words in the translation and untranslated words in the source even in the absence of a reference translation.
arXiv Detail & Related papers (2022-03-03T18:59:02Z) - Uncertainty-Aware Semantic Augmentation for Neural Machine Translation [37.555675157198145]
We propose uncertainty-aware semantic augmentation, which explicitly captures the universal semantic information among multiple semantically-equivalent source sentences.
Our approach significantly outperforms the strong baselines and the existing methods.
arXiv Detail & Related papers (2020-10-09T07:48:09Z) - Language Model Prior for Low-Resource Neural Machine Translation [85.55729693003829]
We propose a novel approach to incorporate a LM as prior in a neural translation model (TM)
We add a regularization term, which pushes the output distributions of the TM to be probable under the LM prior.
Results on two low-resource machine translation datasets show clear improvements even with limited monolingual data.
arXiv Detail & Related papers (2020-04-30T16:29:56Z) - Explicit Reordering for Neural Machine Translation [50.70683739103066]
In Transformer-based neural machine translation (NMT), the positional encoding mechanism helps the self-attention networks to learn the source representation with order dependency.
We propose a novel reordering method to explicitly model this reordering information for the Transformer-based NMT.
The empirical results on the WMT14 English-to-German, WAT ASPEC Japanese-to-English, and WMT17 Chinese-to-English translation tasks show the effectiveness of the proposed approach.
arXiv Detail & Related papers (2020-04-08T05:28:46Z) - A Probabilistic Formulation of Unsupervised Text Style Transfer [128.80213211598752]
We present a deep generative model for unsupervised text style transfer that unifies previously proposed non-generative techniques.
By hypothesizing a parallel latent sequence that generates each observed sequence, our model learns to transform sequences from one domain to another in a completely unsupervised fashion.
arXiv Detail & Related papers (2020-02-10T16:20:49Z) - Explicit Sentence Compression for Neural Machine Translation [110.98786673598016]
State-of-the-art Transformer-based neural machine translation (NMT) systems still follow a standard encoder-decoder framework.
backbone information, which stands for the gist of a sentence, is not specifically focused on.
We propose an explicit sentence compression method to enhance the source sentence representation for NMT.
arXiv Detail & Related papers (2019-12-27T04:14:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.