The Reality of Multi-Lingual Machine Translation
- URL: http://arxiv.org/abs/2202.12814v1
- Date: Fri, 25 Feb 2022 16:44:06 GMT
- Title: The Reality of Multi-Lingual Machine Translation
- Authors: Tom Kocmi and Dominik Mach\'a\v{c}ek and Ond\v{r}ej Bojar
- Abstract summary: "The Reality of Multi-Lingual Machine Translation" discusses the benefits and perils of using more than two languages in machine translation systems.
Author: Machine translation is for us a prime example of deep learning applications.
- Score: 3.183845608678763
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Our book "The Reality of Multi-Lingual Machine Translation" discusses the
benefits and perils of using more than two languages in machine translation
systems. While focused on the particular task of sequence-to-sequence
processing and multi-task learning, the book targets somewhat beyond the area
of natural language processing. Machine translation is for us a prime example
of deep learning applications where human skills and learning capabilities are
taken as a benchmark that many try to match and surpass. We document that some
of the gains observed in multi-lingual translation may result from simpler
effects than the assumed cross-lingual transfer of knowledge.
In the first, rather general part, the book will lead you through the
motivation for multi-linguality, the versatility of deep neural networks
especially in sequence-to-sequence tasks to complications of this learning. We
conclude the general part with warnings against too optimistic and unjustified
explanations of the gains that neural networks demonstrate.
In the second part, we fully delve into multi-lingual models, with a
particularly careful examination of transfer learning as one of the more
straightforward approaches utilizing additional languages. The recent
multi-lingual techniques, including massive models, are surveyed and practical
aspects of deploying systems for many languages are discussed. The conclusion
highlights the open problem of machine understanding and reminds of two ethical
aspects of building large-scale models: the inclusivity of research and its
ecological trace.
Related papers
- Informative Language Representation Learning for Massively Multilingual
Neural Machine Translation [47.19129812325682]
In a multilingual neural machine translation model, an artificial language token is usually used to guide translation into the desired target language.
Recent studies show that prepending language tokens sometimes fails to navigate the multilingual neural machine translation models into right translation directions.
We propose two methods, language embedding embodiment and language-aware multi-head attention, to learn informative language representations to channel translation into right directions.
arXiv Detail & Related papers (2022-09-04T04:27:17Z) - On the cross-lingual transferability of multilingual prototypical models
across NLU tasks [2.44288434255221]
Supervised deep learning-based approaches have been applied to task-oriented dialog and have proven to be effective for limited domain and language applications.
In practice, these approaches suffer from the drawbacks of domain-driven design and under-resourced languages.
This article proposes to investigate the cross-lingual transferability of using synergistically few-shot learning with prototypical neural networks and multilingual Transformers-based models.
arXiv Detail & Related papers (2022-07-19T09:55:04Z) - Towards Best Practices for Training Multilingual Dense Retrieval Models [54.91016739123398]
We focus on the task of monolingual retrieval in a variety of typologically diverse languages using one such design.
Our study is organized as a "best practices" guide for training multilingual dense retrieval models.
arXiv Detail & Related papers (2022-04-05T17:12:53Z) - Exploring Teacher-Student Learning Approach for Multi-lingual
Speech-to-Intent Classification [73.5497360800395]
We develop an end-to-end system that supports multiple languages.
We exploit knowledge from a pre-trained multi-lingual natural language processing model.
arXiv Detail & Related papers (2021-09-28T04:43:11Z) - Importance-based Neuron Allocation for Multilingual Neural Machine
Translation [27.65375150324557]
We propose to divide the model neurons into general and language-specific parts based on their importance across languages.
The general part is responsible for preserving the general knowledge and participating in the translation of all the languages.
The language-specific part is responsible for preserving the language-specific knowledge and participating in the translation of some specific languages.
arXiv Detail & Related papers (2021-07-14T09:15:05Z) - AM2iCo: Evaluating Word Meaning in Context across Low-ResourceLanguages
with Adversarial Examples [51.048234591165155]
We present AM2iCo, Adversarial and Multilingual Meaning in Context.
It aims to faithfully assess the ability of state-of-the-art (SotA) representation models to understand the identity of word meaning in cross-lingual contexts.
Results reveal that current SotA pretrained encoders substantially lag behind human performance.
arXiv Detail & Related papers (2021-04-17T20:23:45Z) - First Align, then Predict: Understanding the Cross-Lingual Ability of
Multilingual BERT [2.2931318723689276]
Cross-lingual transfer emerges from fine-tuning on a task of interest in one language and evaluating on a distinct language, not seen during the fine-tuning.
We show that multilingual BERT can be viewed as the stacking of two sub-networks: a multilingual encoder followed by a task-specific language-agnostic predictor.
While the encoder is crucial for cross-lingual transfer and remains mostly unchanged during fine-tuning, the task predictor has little importance on the transfer and can be red during fine-tuning.
arXiv Detail & Related papers (2021-01-26T22:12:38Z) - Evaluating Multilingual Text Encoders for Unsupervised Cross-Lingual
Retrieval [51.60862829942932]
We present a systematic empirical study focused on the suitability of the state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks.
For sentence-level CLIR, we demonstrate that state-of-the-art performance can be achieved.
However, the peak performance is not met using the general-purpose multilingual text encoders off-the-shelf', but rather relying on their variants that have been further specialized for sentence understanding tasks.
arXiv Detail & Related papers (2021-01-21T00:15:38Z) - Bridging Linguistic Typology and Multilingual Machine Translation with
Multi-View Language Representations [83.27475281544868]
We use singular vector canonical correlation analysis to study what kind of information is induced from each source.
We observe that our representations embed typology and strengthen correlations with language relationships.
We then take advantage of our multi-view language vector space for multilingual machine translation, where we achieve competitive overall translation accuracy.
arXiv Detail & Related papers (2020-04-30T16:25:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.