Graph Neural Networks for Multiparallel Word Alignment

• URL: http://arxiv.org/abs/2203.08654v1
• Date: Wed, 16 Mar 2022 14:41:35 GMT
• Title: Graph Neural Networks for Multiparallel Word Alignment
• Authors: Ayyoob Imani, L\"utfi Kerem \c{S}enel, Masoud Jalili Sabet, Fran\c{c}ois Yvon, Hinrich Sch\"utze
• Abstract summary: We compute high-quality word alignments between multiple language pairs by considering all language pairs together. We use graph neural networks (GNNs) to exploit the graph structure. Our method outperforms previous work on three word-alignment datasets and on a downstream task.
• Score: 0.27998963147546146
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: After a period of decrease, interest in word alignments is increasing again for their usefulness in domains such as typological research, cross-lingual annotation projection, and machine translation. Generally, alignment algorithms only use bitext and do not make use of the fact that many parallel corpora are multiparallel. Here, we compute high-quality word alignments between multiple language pairs by considering all language pairs together. First, we create a multiparallel word alignment graph, joining all bilingual word alignment pairs in one graph. Next, we use graph neural networks (GNNs) to exploit the graph structure. Our GNN approach (i) utilizes information about the meaning, position, and language of the input words, (ii) incorporates information from multiple parallel sentences, (iii) adds and removes edges from the initial alignments, and (iv) yields a prediction model that can generalize beyond the training sentences. We show that community detection provides valuable information for multiparallel word alignment. Our method outperforms previous work on three word-alignment datasets and on a downstream task.

Related papers

• Third-Party Aligner for Neural Word Alignments [18.745852103348845]
  We propose to use word alignments generated by a third-party word aligner to supervise the neural word alignment training. Experiments show that our approach can surprisingly do self-correction over the third-party supervision. We achieve state-of-the-art word alignment performances, with averagely more than two points lower alignment error rates than the best third-party aligner.
  arXiv  Detail & Related papers  (2022-11-08T12:30:08Z)
• Graph Algorithms for Multiparallel Word Alignment [2.5200727733264663]
  In this work, we exploit the multiparallelity of corpora by representing an initial set of bilingual alignments as a graph. We present two graph algorithms for edge prediction: one inspired by recommender systems and one based on network link prediction.
  arXiv  Detail & Related papers  (2021-09-13T19:40:29Z)
• Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment [49.45399359826453]
  Cross-lingual language models are typically pretrained with language modeling on multilingual text or parallel sentences. We introduce denoising word alignment as a new cross-lingual pre-training task. Experimental results show that our method improves cross-lingual transferability on various datasets.
  arXiv  Detail & Related papers  (2021-06-11T13:36:01Z)
• Word Alignment by Fine-tuning Embeddings on Parallel Corpora [96.28608163701055]
  Word alignment over parallel corpora has a wide variety of applications, including learning translation lexicons, cross-lingual transfer of language processing tools, and automatic evaluation or analysis of translation outputs. Recently, other work has demonstrated that pre-trained contextualized word embeddings derived from multilingually trained language models (LMs) prove an attractive alternative, achieving competitive results on the word alignment task even in the absence of explicit training on parallel data. In this paper, we examine methods to marry the two approaches: leveraging pre-trained LMs but fine-tuning them on parallel text with objectives designed to improve alignment quality, and proposing
  arXiv  Detail & Related papers  (2021-01-20T17:54:47Z)
• Learning Contextualised Cross-lingual Word Embeddings and Alignments for Extremely Low-Resource Languages Using Parallel Corpora [63.5286019659504]
  We propose a new approach for learning contextualised cross-lingual word embeddings based on a small parallel corpus. Our method obtains word embeddings via an LSTM encoder-decoder model that simultaneously translates and reconstructs an input sentence.
  arXiv  Detail & Related papers  (2020-10-27T22:24:01Z)
• Graph Optimal Transport for Cross-Domain Alignment [121.80313648519203]
  Cross-domain alignment is fundamental to computer vision and natural language processing. We propose Graph Optimal Transport (GOT), a principled framework that germinates from recent advances in Optimal Transport (OT) Experiments show consistent outperformance of GOT over baselines across a wide range of tasks.
  arXiv  Detail & Related papers  (2020-06-26T01:14:23Z)
• Filtered Inner Product Projection for Crosslingual Embedding Alignment [28.72288652451881]
  Filtered Inner Product Projection (FIPP) is a method for mapping embeddings to a common representation space. FIPP is applicable even when the source and target embeddings are of differing dimensionalities. We show that our approach outperforms existing methods on the MUSE dataset for various language pairs.
  arXiv  Detail & Related papers  (2020-06-05T19:53:30Z)
• SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings [3.8424737607413153]
  We propose word alignment methods that require no parallel data. Key idea is to leverage multilingual word embeddings, both static and contextualized, for word alignment. We find that alignments created from embeddings are superior for two language pairs compared to those produced by traditional statistical methods.
  arXiv  Detail & Related papers  (2020-04-18T23:10:36Z)
• On the Language Neutrality of Pre-trained Multilingual Representations [70.93503607755055]
  We investigate the language-neutrality of multilingual contextual embeddings directly and with respect to lexical semantics. Our results show that contextual embeddings are more language-neutral and, in general, more informative than aligned static word-type embeddings. We show how to reach state-of-the-art accuracy on language identification and match the performance of statistical methods for word alignment of parallel sentences.
  arXiv  Detail & Related papers  (2020-04-09T19:50:32Z)
• Multilingual Alignment of Contextual Word Representations [49.42244463346612]
  BERT exhibits significantly improved zero-shot performance on XNLI compared to the base model. We introduce a contextual version of word retrieval and show that it correlates well with downstream zero-shot transfer. These results support contextual alignment as a useful concept for understanding large multilingual pre-trained models.
  arXiv  Detail & Related papers  (2020-02-10T03:27:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.