Third-Party Aligner for Neural Word Alignments

• URL: http://arxiv.org/abs/2211.04198v1
• Date: Tue, 8 Nov 2022 12:30:08 GMT
• Title: Third-Party Aligner for Neural Word Alignments
• Authors: Jinpeng Zhang, Chuanqi Dong, Xiangyu Duan, Yuqi Zhang, Min Zhang
• Abstract summary: We propose to use word alignments generated by a third-party word aligner to supervise the neural word alignment training. Experiments show that our approach can surprisingly do self-correction over the third-party supervision. We achieve state-of-the-art word alignment performances, with averagely more than two points lower alignment error rates than the best third-party aligner.
• Score: 18.745852103348845
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Word alignment is to find translationally equivalent words between source and target sentences. Previous work has demonstrated that self-training can achieve competitive word alignment results. In this paper, we propose to use word alignments generated by a third-party word aligner to supervise the neural word alignment training. Specifically, source word and target word of each word pair aligned by the third-party aligner are trained to be close neighbors to each other in the contextualized embedding space when fine-tuning a pre-trained cross-lingual language model. Experiments on the benchmarks of various language pairs show that our approach can surprisingly do self-correction over the third-party supervision by finding more accurate word alignments and deleting wrong word alignments, leading to better performance than various third-party word aligners, including the currently best one. When we integrate all supervisions from various third-party aligners, we achieve state-of-the-art word alignment performances, with averagely more than two points lower alignment error rates than the best third-party aligner. We released our code at https://github.com/sdongchuanqi/Third-Party-Supervised-Aligner.

Related papers

• How Transliterations Improve Crosslingual Alignment [48.929677368744606]
  Recent studies have shown that post-aligning multilingual pretrained language models (mPLMs) using alignment objectives can improve crosslingual alignment. This paper attempts to explicitly evaluate the crosslingual alignment and identify the key elements in transliteration-based approaches that contribute to better performance.
  arXiv  Detail & Related papers  (2024-09-25T20:05:45Z)
• Unbalanced Optimal Transport for Unbalanced Word Alignment [17.08341136230076]
  This study shows that the family of optimal transport (OT), i.e. balanced, partial, and unbalanced OT, are natural and powerful approaches even without tailor-made techniques. Our experiments covering unsupervised and supervised settings indicate that our generic OT-based alignment methods are competitive against the state-of-the-arts specially designed for word alignment.
  arXiv  Detail & Related papers  (2023-06-07T03:03:41Z)
• Graph Neural Networks for Multiparallel Word Alignment [0.27998963147546146]
  We compute high-quality word alignments between multiple language pairs by considering all language pairs together. We use graph neural networks (GNNs) to exploit the graph structure. Our method outperforms previous work on three word-alignment datasets and on a downstream task.
  arXiv  Detail & Related papers  (2022-03-16T14:41:35Z)
• Simple, Interpretable and Stable Method for Detecting Words with Usage Change across Corpora [54.757845511368814]
  The problem of comparing two bodies of text and searching for words that differ in their usage arises often in digital humanities and computational social science. This is commonly approached by training word embeddings on each corpus, aligning the vector spaces, and looking for words whose cosine distance in the aligned space is large. We propose an alternative approach that does not use vector space alignment, and instead considers the neighbors of each word.
  arXiv  Detail & Related papers  (2021-12-28T23:46:00Z)
• Using Optimal Transport as Alignment Objective for fine-tuning Multilingual Contextualized Embeddings [7.026476782041066]
  We propose using Optimal Transport (OT) as an alignment objective during fine-tuning to improve multilingual contextualized representations. This approach does not require word-alignment pairs prior to fine-tuning and instead learns the word alignments within context in an unsupervised manner.
  arXiv  Detail & Related papers  (2021-10-06T16:13:45Z)
• SLUA: A Super Lightweight Unsupervised Word Alignment Model via Cross-Lingual Contrastive Learning [79.91678610678885]
  We propose a super lightweight unsupervised word alignment model (SLUA) Experimental results on several public benchmarks demonstrate that our model achieves competitive, if not better, performance. Notably, we recognize our model as a pioneer attempt to unify bilingual word embedding and word alignments.
  arXiv  Detail & Related papers  (2021-02-08T05:54:11Z)
• Word Alignment by Fine-tuning Embeddings on Parallel Corpora [96.28608163701055]
  Word alignment over parallel corpora has a wide variety of applications, including learning translation lexicons, cross-lingual transfer of language processing tools, and automatic evaluation or analysis of translation outputs. Recently, other work has demonstrated that pre-trained contextualized word embeddings derived from multilingually trained language models (LMs) prove an attractive alternative, achieving competitive results on the word alignment task even in the absence of explicit training on parallel data. In this paper, we examine methods to marry the two approaches: leveraging pre-trained LMs but fine-tuning them on parallel text with objectives designed to improve alignment quality, and proposing
  arXiv  Detail & Related papers  (2021-01-20T17:54:47Z)
• Mask-Align: Self-Supervised Neural Word Alignment [47.016975106231875]
  Mask-Align is a self-supervised model specifically designed for the word alignment task. Our model parallelly masks and predicts each target token, and extracts high-quality alignments without any supervised loss.
  arXiv  Detail & Related papers  (2020-12-13T21:44:29Z)
• A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERT [22.701728185474195]
  We first formalize a word alignment problem as a collection of independent predictions from a token in the source sentence to a span in the target sentence. We then solve this problem by using multilingual BERT, which is fine-tuned on a manually created gold word alignment data. We show that the proposed method significantly outperformed previous supervised and unsupervised word alignment methods without using any bitexts for pretraining.
  arXiv  Detail & Related papers  (2020-04-29T23:40:08Z)
• Multilingual Alignment of Contextual Word Representations [49.42244463346612]
  BERT exhibits significantly improved zero-shot performance on XNLI compared to the base model. We introduce a contextual version of word retrieval and show that it correlates well with downstream zero-shot transfer. These results support contextual alignment as a useful concept for understanding large multilingual pre-trained models.
  arXiv  Detail & Related papers  (2020-02-10T03:27:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.