Unbalanced Optimal Transport for Unbalanced Word Alignment

• URL: http://arxiv.org/abs/2306.04116v1
• Date: Wed, 7 Jun 2023 03:03:41 GMT
• Title: Unbalanced Optimal Transport for Unbalanced Word Alignment
• Authors: Yuki Arase, Han Bao, Sho Yokoi
• Abstract summary: This study shows that the family of optimal transport (OT), i.e. balanced, partial, and unbalanced OT, are natural and powerful approaches even without tailor-made techniques. Our experiments covering unsupervised and supervised settings indicate that our generic OT-based alignment methods are competitive against the state-of-the-arts specially designed for word alignment.
• Score: 17.08341136230076
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Monolingual word alignment is crucial to model semantic interactions between sentences. In particular, null alignment, a phenomenon in which words have no corresponding counterparts, is pervasive and critical in handling semantically divergent sentences. Identification of null alignment is useful on its own to reason about the semantic similarity of sentences by indicating there exists information inequality. To achieve unbalanced word alignment that values both alignment and null alignment, this study shows that the family of optimal transport (OT), i.e., balanced, partial, and unbalanced OT, are natural and powerful approaches even without tailor-made techniques. Our extensive experiments covering unsupervised and supervised settings indicate that our generic OT-based alignment methods are competitive against the state-of-the-arts specially designed for word alignment, remarkably on challenging datasets with high null alignment frequencies.

Related papers

• How Transliterations Improve Crosslingual Alignment [48.929677368744606]
  Recent studies have shown that post-aligning multilingual pretrained language models (mPLMs) using alignment objectives can improve crosslingual alignment. This paper attempts to explicitly evaluate the crosslingual alignment and identify the key elements in transliteration-based approaches that contribute to better performance.
  arXiv  Detail & Related papers  (2024-09-25T20:05:45Z)
• RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank [54.854714257687334]
  We propose a novel approach, RankCSE, for unsupervised sentence representation learning. It incorporates ranking consistency and ranking distillation with contrastive learning into a unified framework. An extensive set of experiments are conducted on both semantic textual similarity (STS) and transfer (TR) tasks.
  arXiv  Detail & Related papers  (2023-05-26T08:27:07Z)
• Third-Party Aligner for Neural Word Alignments [18.745852103348845]
  We propose to use word alignments generated by a third-party word aligner to supervise the neural word alignment training. Experiments show that our approach can surprisingly do self-correction over the third-party supervision. We achieve state-of-the-art word alignment performances, with averagely more than two points lower alignment error rates than the best third-party aligner.
  arXiv  Detail & Related papers  (2022-11-08T12:30:08Z)
• Non-Monotonic Latent Alignments for CTC-Based Non-Autoregressive Machine Translation [15.309573393914462]
  Non-autoregressive translation (NAT) models are typically trained with the cross-entropy loss. Latent alignment models relax the explicit alignment by marginalizing out all monotonic latent alignments with the CTC loss. We extend the alignment space to non-monotonic alignments to allow for the global word reordering.
  arXiv  Detail & Related papers  (2022-10-08T07:44:28Z)
• Keywords and Instances: A Hierarchical Contrastive Learning Framework Unifying Hybrid Granularities for Text Generation [59.01297461453444]
  We propose a hierarchical contrastive learning mechanism, which can unify hybrid granularities semantic meaning in the input text. Experiments demonstrate that our model outperforms competitive baselines on paraphrasing, dialogue generation, and storytelling tasks.
  arXiv  Detail & Related papers  (2022-05-26T13:26:03Z)
• Accurate Online Posterior Alignments for Principled Lexically-Constrained Decoding [40.212186465135304]
  We propose a novel posterior alignment technique that is truly online in its execution and superior in terms of alignment error rates. On five language pairs, including two distant language pairs, we achieve consistent drop in alignment error rates. When deployed on seven lexically constrained translation tasks, we achieve significant improvements in BLEU specifically around the constrained positions.
  arXiv  Detail & Related papers  (2022-04-02T14:37:07Z)
• Automatically Identifying Semantic Bias in Crowdsourced Natural Language Inference Datasets [78.6856732729301]
  We introduce a model-driven, unsupervised technique to find "bias clusters" in a learned embedding space of hypotheses in NLI datasets. interventions and additional rounds of labeling can be performed to ameliorate the semantic bias of the hypothesis distribution of a dataset.
  arXiv  Detail & Related papers  (2021-12-16T22:49:01Z)
• Contextualized Semantic Distance between Highly Overlapped Texts [85.1541170468617]
  Overlapping frequently occurs in paired texts in natural language processing tasks like text editing and semantic similarity evaluation. This paper aims to address the issue with a mask-and-predict strategy. We take the words in the longest common sequence as neighboring words and use masked language modeling (MLM) to predict the distributions on their positions. Experiments on Semantic Textual Similarity show NDD to be more sensitive to various semantic differences, especially on highly overlapped paired texts.
  arXiv  Detail & Related papers  (2021-10-04T03:59:15Z)
• Leveraging Neural Machine Translation for Word Alignment [0.0]
  A machine translation (MT) system is able to produce word-alignments using the trained attention heads. This is convenient because word-alignment is theoretically a viable byproduct of any attention-based NMT. We summarize different approaches on how word-alignment can be extracted from alignment scores and then explore ways in which scores can be extracted from NMT.
  arXiv  Detail & Related papers  (2021-03-31T17:51:35Z)
• SLUA: A Super Lightweight Unsupervised Word Alignment Model via Cross-Lingual Contrastive Learning [79.91678610678885]
  We propose a super lightweight unsupervised word alignment model (SLUA) Experimental results on several public benchmarks demonstrate that our model achieves competitive, if not better, performance. Notably, we recognize our model as a pioneer attempt to unify bilingual word embedding and word alignments.
  arXiv  Detail & Related papers  (2021-02-08T05:54:11Z)
• Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport [14.86310501896212]
  In this work, we extend this selective rationalization approach to text matching. The goal is to jointly select and align text pieces, such as tokens or sentences, as a justification for the downstream prediction. Our approach employs optimal transport (OT) to find a minimal cost alignment between the inputs.
  arXiv  Detail & Related papers  (2020-05-27T01:20:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.