Unbalanced Optimal Transport for Unbalanced Word Alignment

• URL: http://arxiv.org/abs/2306.04116v1
• Date: Wed, 7 Jun 2023 03:03:41 GMT
• Title: Unbalanced Optimal Transport for Unbalanced Word Alignment
• Abstract summary: This study shows that the family of optimal transport (OT), i.e. balanced, partial, and unbalanced OT, are natural and powerful approaches even without tailor-made techniques. Our experiments covering unsupervised and supervised settings indicate that our generic OT-based alignment methods are competitive against the state-of-the-arts specially designed for word alignment.
• Score: 17.08341136230076
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Monolingual word alignment is crucial to model semantic interactions between sentences. In particular, null alignment, a phenomenon in which words have no corresponding counterparts, is pervasive and critical in handling semantically divergent sentences. Identification of null alignment is useful on its own to reason about the semantic similarity of sentences by indicating there exists information inequality. To achieve unbalanced word alignment that values both alignment and null alignment, this study shows that the family of optimal transport (OT), i.e., balanced, partial, and unbalanced OT, are natural and powerful approaches even without tailor-made techniques. Our extensive experiments covering unsupervised and supervised settings indicate that our generic OT-based alignment methods are competitive against the state-of-the-arts specially designed for word alignment, remarkably on challenging datasets with high null alignment frequencies.

Related papers

• OmniAlign: A Unified Multilingual Aligner for Word and Sentence Alignment [10.629976569255229]
  We present OmniAlign, a unified multilingual aligner that supports both word-level and sentence-level alignment.<n>Built on an encoder-only backbone with strong long-context modeling, OmniAlign induces word alignments from contextualized token similarity matrices.<n>Experiments show that OmniAlign achieves highly competitive performance on both word- and sentence-alignment benchmarks.
  arXiv  Detail & Related papers  (2026-08-19T03:02:43Z)
• Rank-Aware Hyperbolic Alignment for Vision-Language Dataset Distillation [54.74045834035952]
  Vision-language dataset distillation (VLDD) compresses a large image-text paired dataset into a small set of synthetic pairs.<n>We propose a rank-aware hyperbolic alignment (RAHA) that combines hierarchical geometry with explicit alignment-capacity control.<n>RAHA demonstrates competitive cross-modal retrieval and improved transfer indicators under fixed budgets.
  arXiv  Detail & Related papers  (2026-06-28T15:41:31Z)
• Why Better Cross-Lingual Alignment Fails for Better Cross-Lingual Transfer: Case of Encoders [49.098124747769276]
  We show that explicit alignment techniques frequently fail to improve token-level downstream performance.<n>We analyze four XLM-R encoder models aligned on different language pairs and fine-tuned for either POS Tagging or Sentence Classification.
  arXiv  Detail & Related papers  (2026-03-19T13:10:39Z)
• Enhancing CLIP Robustness via Cross-Modality Alignment [54.01929554563447]
  We propose Cross-modality Alignment, an optimal transport-based framework for vision-language models.<n> COLA restores global image-text alignment and local structural consistency in the feature space.<n> COLA is training-free and compatible with existing fine-tuned models.
  arXiv  Detail & Related papers  (2025-10-28T03:47:44Z)
• Cross-modal Full-mode Fine-grained Alignment for Text-to-Image Person Retrieval [54.90229711181207]
  Text-to-Image Person Retrieval (TIPR) aims to retrieve the most relevant person images based on a given text query.<n>The key challenge in TIPR lies in achieving effective alignment between textual and visual modalities.<n>We propose FMFA, a cross-modal Full-Mode Fine-grained Alignment framework.
  arXiv  Detail & Related papers  (2025-09-17T07:12:05Z)
• How Transliterations Improve Crosslingual Alignment [48.929677368744606]
  Recent studies have shown that post-aligning multilingual pretrained language models (mPLMs) using alignment objectives can improve crosslingual alignment. This paper attempts to explicitly evaluate the crosslingual alignment and identify the key elements in transliteration-based approaches that contribute to better performance.
  arXiv  Detail & Related papers  (2024-09-25T20:05:45Z)
• Combating Confirmation Bias: A Unified Pseudo-Labeling Framework for Entity Alignment [30.407534668054286]
  We propose a Unified Pseudo-Labeling framework for Entity Alignment (UPL-EA)<n>UPL-EA explicitly eliminates pseudo-labeling errors to boost the accuracy of entity alignment.<n>Our results and in-depth analyses demonstrate the superiority of UPL-EA over 15 competitive baselines.
  arXiv  Detail & Related papers  (2023-07-05T07:32:34Z)
• RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank [54.854714257687334]
  We propose a novel approach, RankCSE, for unsupervised sentence representation learning. It incorporates ranking consistency and ranking distillation with contrastive learning into a unified framework. An extensive set of experiments are conducted on both semantic textual similarity (STS) and transfer (TR) tasks.
  arXiv  Detail & Related papers  (2023-05-26T08:27:07Z)
• Third-Party Aligner for Neural Word Alignments [18.745852103348845]
  We propose to use word alignments generated by a third-party word aligner to supervise the neural word alignment training. Experiments show that our approach can surprisingly do self-correction over the third-party supervision. We achieve state-of-the-art word alignment performances, with averagely more than two points lower alignment error rates than the best third-party aligner.
  arXiv  Detail & Related papers  (2022-11-08T12:30:08Z)
• Non-Monotonic Latent Alignments for CTC-Based Non-Autoregressive Machine Translation [15.309573393914462]
  Non-autoregressive translation (NAT) models are typically trained with the cross-entropy loss. Latent alignment models relax the explicit alignment by marginalizing out all monotonic latent alignments with the CTC loss. We extend the alignment space to non-monotonic alignments to allow for the global word reordering.
  arXiv  Detail & Related papers  (2022-10-08T07:44:28Z)
• Keywords and Instances: A Hierarchical Contrastive Learning Framework Unifying Hybrid Granularities for Text Generation [59.01297461453444]
  We propose a hierarchical contrastive learning mechanism, which can unify hybrid granularities semantic meaning in the input text. Experiments demonstrate that our model outperforms competitive baselines on paraphrasing, dialogue generation, and storytelling tasks.
  arXiv  Detail & Related papers  (2022-05-26T13:26:03Z)
• Accurate Online Posterior Alignments for Principled Lexically-Constrained Decoding [40.212186465135304]
  We propose a novel posterior alignment technique that is truly online in its execution and superior in terms of alignment error rates. On five language pairs, including two distant language pairs, we achieve consistent drop in alignment error rates. When deployed on seven lexically constrained translation tasks, we achieve significant improvements in BLEU specifically around the constrained positions.
  arXiv  Detail & Related papers  (2022-04-02T14:37:07Z)
• Automatically Identifying Semantic Bias in Crowdsourced Natural Language Inference Datasets [78.6856732729301]
  We introduce a model-driven, unsupervised technique to find "bias clusters" in a learned embedding space of hypotheses in NLI datasets. interventions and additional rounds of labeling can be performed to ameliorate the semantic bias of the hypothesis distribution of a dataset.
  arXiv  Detail & Related papers  (2021-12-16T22:49:01Z)
• Contextualized Semantic Distance between Highly Overlapped Texts [85.1541170468617]
  Overlapping frequently occurs in paired texts in natural language processing tasks like text editing and semantic similarity evaluation. This paper aims to address the issue with a mask-and-predict strategy. We take the words in the longest common sequence as neighboring words and use masked language modeling (MLM) to predict the distributions on their positions. Experiments on Semantic Textual Similarity show NDD to be more sensitive to various semantic differences, especially on highly overlapped paired texts.
  arXiv  Detail & Related papers  (2021-10-04T03:59:15Z)
• Leveraging Neural Machine Translation for Word Alignment [0.0]
  A machine translation (MT) system is able to produce word-alignments using the trained attention heads. This is convenient because word-alignment is theoretically a viable byproduct of any attention-based NMT. We summarize different approaches on how word-alignment can be extracted from alignment scores and then explore ways in which scores can be extracted from NMT.
  arXiv  Detail & Related papers  (2021-03-31T17:51:35Z)
• SLUA: A Super Lightweight Unsupervised Word Alignment Model via Cross-Lingual Contrastive Learning [79.91678610678885]
  We propose a super lightweight unsupervised word alignment model (SLUA) Experimental results on several public benchmarks demonstrate that our model achieves competitive, if not better, performance. Notably, we recognize our model as a pioneer attempt to unify bilingual word embedding and word alignments.
  arXiv  Detail & Related papers  (2021-02-08T05:54:11Z)
• Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport [14.86310501896212]
  In this work, we extend this selective rationalization approach to text matching. The goal is to jointly select and align text pieces, such as tokens or sentences, as a justification for the downstream prediction. Our approach employs optimal transport (OT) to find a minimal cost alignment between the inputs.
  arXiv  Detail & Related papers  (2020-05-27T01:20:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.