Dual Reconstruction: a Unifying Objective for Semi-Supervised Neural Machine Translation

• URL: http://arxiv.org/abs/2010.03412v1
• Date: Wed, 7 Oct 2020 13:40:32 GMT
• Title: Dual Reconstruction: a Unifying Objective for Semi-Supervised Neural Machine Translation
• Abstract summary: Iterative Back-Translation and Dual Learning effectively incorporate monolingual training data in neural machine translation, they use different objectives and gradient approximation strategies, and have not been extensively compared. We introduce a novel dual reconstruction objective that provides a unified view of Iterative Back-Translation and Dual Learning.
• Score: 32.99075556718486
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While Iterative Back-Translation and Dual Learning effectively incorporate monolingual training data in neural machine translation, they use different objectives and heuristic gradient approximation strategies, and have not been extensively compared. We introduce a novel dual reconstruction objective that provides a unified view of Iterative Back-Translation and Dual Learning. It motivates a theoretical analysis and controlled empirical study on German-English and Turkish-English tasks, which both suggest that Iterative Back-Translation is more effective than Dual Learning despite its relative simplicity.

Related papers

• ReflectMT: Internalizing Reflection for Efficient and High-Quality Machine Translation [63.89730738735446]
  We propose ReflectMT, a two-stage reflection internalization algorithm for machine translation.<n>Our approach develops the model's "translate-reflect-refine" capability through reinforcement learning.<n> Experimental results on datasets such as WMT24 demonstrate that our model's first-pass translations during inference outperform multi-step reasoning LRMs.
  arXiv  Detail & Related papers  (2026-04-21T06:48:41Z)
• LLM-based Translation Inference with Iterative Bilingual Understanding [52.46978502902928]
  We propose a novel Iterative Bilingual Understanding Translation method based on the cross-lingual capabilities of large language models (LLMs) The cross-lingual capability of LLMs enables the generation of contextual understanding for both the source and target languages separately. The proposed IBUT outperforms several strong comparison methods.
  arXiv  Detail & Related papers  (2024-10-16T13:21:46Z)
• DUAL-REFLECT: Enhancing Large Language Models for Reflective Translation through Dual Learning Feedback Mechanisms [43.148203559785095]
  Large language models (LLMs) enhanced by self-reflection have achieved promising performance on machine translation. Existing self-reflection methods lack effective feedback information, limiting the translation performance. We introduce aREFLECT framework, leveraging the dual learning of translation tasks to provide effective feedback.
  arXiv  Detail & Related papers  (2024-06-11T13:10:39Z)
• Detection and Positive Reconstruction of Cognitive Distortion sentences: Mandarin Dataset and Evaluation [22.070611256611635]
  This research introduces a Positive Reconstruction Framework based on positive psychology theory. We provide a corpus containing 4001 instances for detecting cognitive distortions and 1900 instances for positive reconstruction in Mandarin.
  arXiv  Detail & Related papers  (2024-05-24T08:17:20Z)
• Neural Machine Translation with Contrastive Translation Memories [71.86990102704311]
  Retrieval-augmented Neural Machine Translation models have been successful in many translation scenarios. We propose a new retrieval-augmented NMT to model contrastively retrieved translation memories that are holistically similar to the source sentence. In training phase, a Multi-TM contrastive learning objective is introduced to learn salient feature of each TM with respect to target sentence.
  arXiv  Detail & Related papers  (2022-12-06T17:10:17Z)
• Contrastive Instruction-Trajectory Learning for Vision-Language Navigation [66.16980504844233]
  A vision-language navigation (VLN) task requires an agent to reach a target with the guidance of natural language instruction. Previous works fail to discriminate the similarities and discrepancies across instruction-trajectory pairs and ignore the temporal continuity of sub-instructions. We propose a Contrastive Instruction-Trajectory Learning framework that explores invariance across similar data samples and variance across different ones to learn distinctive representations for robust navigation.
  arXiv  Detail & Related papers  (2021-12-08T06:32:52Z)
• Machine Translation of Low-Resource Indo-European Languages [7.113332213566949]
  We build two systems to study how relatedness can benefit the translation performance. The primary system adopts machine translation model pre-trained on related language pair and the contrastive system adopts that pre-trained on unrelated language pair.
  arXiv  Detail & Related papers  (2021-08-08T21:41:08Z)
• Two-Way Neural Machine Translation: A Proof of Concept for Bidirectional Translation Modeling using a Two-Dimensional Grid [47.39346022004215]
  This paper proposes to build a single end-to-end bidirectional translation model using a two-dimensional grid. Instead of training two models independently, our approach encourages a single network to jointly learn to translate in both directions.
  arXiv  Detail & Related papers  (2020-11-24T15:42:32Z)
• Self-Paced Learning for Neural Machine Translation [55.41314278859938]
  We propose self-paced learning for neural machine translation (NMT) training. We show that the proposed model yields better performance than strong baselines.
  arXiv  Detail & Related papers  (2020-10-09T11:33:16Z)
• A Deep Reinforced Model for Zero-Shot Cross-Lingual Summarization with Bilingual Semantic Similarity Rewards [40.17497211507507]
  Cross-lingual text summarization is a practically important but under-explored task. We propose an end-to-end cross-lingual text summarization model.
  arXiv  Detail & Related papers  (2020-06-27T21:51:38Z)
• Dual Learning: Theoretical Study and an Algorithmic Extension [168.0485041314289]
  We propose multi-step dual learning, in which we leverage feedback signals from additional domains to improve the qualities of mappings. We prove that multi-step dual learn-ing can boost the performance of standard dual learning under mild conditions.
  arXiv  Detail & Related papers  (2020-05-17T12:14:35Z)
• Translation Artifacts in Cross-lingual Transfer Learning [51.66536640084888]
  We show that machine translation can introduce subtle artifacts that have a notable impact in existing cross-lingual models. In natural language inference, translating the premise and the hypothesis independently can reduce the lexical overlap between them. We also improve the state-of-the-art in XNLI for the translate-test and zero-shot approaches by 4.3 and 2.8 points, respectively.
  arXiv  Detail & Related papers  (2020-04-09T17:54:30Z)
• Reference Language based Unsupervised Neural Machine Translation [108.64894168968067]
  unsupervised neural machine translation almost completely relieves the parallel corpus curse. We propose a new reference language-based framework for UNMT, RUNMT, in which the reference language only shares a parallel corpus with the source. Experimental results show that our methods improve the quality of UNMT over that of a strong baseline that uses only one auxiliary language.
  arXiv  Detail & Related papers  (2020-04-05T08:28:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.