BinaryAlign: Word Alignment as Binary Sequence Labeling

• URL: http://arxiv.org/abs/2407.12881v1
• Date: Tue, 16 Jul 2024 15:11:06 GMT
• Title: BinaryAlign: Word Alignment as Binary Sequence Labeling
• Authors: Gaetan Lopez Latouche, Marc-André Carbonneau, Ben Swanson,
• Abstract summary: We propose BinaryAlign, a novel word alignment technique based on binary sequence labeling. We explore the performance of BinaryAlign on non-English language pairs.
• Score: 2.5575527199248347
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Real world deployments of word alignment are almost certain to cover both high and low resource languages. However, the state-of-the-art for this task recommends a different model class depending on the availability of gold alignment training data for a particular language pair. We propose BinaryAlign, a novel word alignment technique based on binary sequence labeling that outperforms existing approaches in both scenarios, offering a unifying approach to the task. Additionally, we vary the specific choice of multilingual foundation model, perform stratified error analysis over alignment error type, and explore the performance of BinaryAlign on non-English language pairs. We make our source code publicly available.

Related papers

• What Drives Cross-lingual Ranking? Retrieval Approaches with Multilingual Language Models [0.19116784879310025]
  Cross-lingual information retrieval is challenging due to disparities in resources, scripts, and weak cross-lingual semantic alignment in embedding models.<n>Existing pipelines often rely on translation and monolingual retrievals, which add computational overhead and noise, performance.<n>This work systematically evaluates four intervention types, namely document translation, multilingual dense retrieval with pretrained encoders, contrastive learning at word, phrase, and query-document levels, and cross-encoder re-ranking, across three benchmark datasets.
  arXiv  Detail & Related papers  (2025-11-24T17:17:40Z)
• How Good is BLI as an Alignment Measure: A Study in Word Embedding Paradigm [1.4712349476860904]
  We explore the strengths and limitations of BLI as a measure to evaluate the degree of alignment of two embedding spaces.<n>We evaluate how well traditional embedding alignment techniques, novel multilingual models, and combined alignment techniques perform BLI tasks.<n>We propose a novel stem-based BLI approach to evaluate two aligned embedding spaces that take into account the inflected nature of languages.
  arXiv  Detail & Related papers  (2025-11-17T06:41:41Z)
• Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Training [58.696660064190475]
  We find that the existence of code-switching, alternating between different languages within a context, is key to multilingual capabilities. To better explore the power of code-switching for language alignment during pre-training, we investigate the strategy of synthetic code-switching.
  arXiv  Detail & Related papers  (2025-04-02T15:09:58Z)
• How Transliterations Improve Crosslingual Alignment [48.929677368744606]
  Recent studies have shown that post-aligning multilingual pretrained language models (mPLMs) using alignment objectives can improve crosslingual alignment. This paper attempts to explicitly evaluate the crosslingual alignment and identify the key elements in transliteration-based approaches that contribute to better performance.
  arXiv  Detail & Related papers  (2024-09-25T20:05:45Z)
• Dual-Alignment Pre-training for Cross-lingual Sentence Embedding [79.98111074307657]
  We propose a dual-alignment pre-training (DAP) framework for cross-lingual sentence embedding. We introduce a novel representation translation learning (RTL) task, where the model learns to use one-side contextualized token representation to reconstruct its translation counterpart. Our approach can significantly improve sentence embedding.
  arXiv  Detail & Related papers  (2023-05-16T03:53:30Z)
• Multi-level Contrastive Learning for Cross-lingual Spoken Language Understanding [90.87454350016121]
  We develop novel code-switching schemes to generate hard negative examples for contrastive learning at all levels. We develop a label-aware joint model to leverage label semantics for cross-lingual knowledge transfer.
  arXiv  Detail & Related papers  (2022-05-07T13:44:28Z)
• Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment [49.45399359826453]
  Cross-lingual language models are typically pretrained with language modeling on multilingual text or parallel sentences. We introduce denoising word alignment as a new cross-lingual pre-training task. Experimental results show that our method improves cross-lingual transferability on various datasets.
  arXiv  Detail & Related papers  (2021-06-11T13:36:01Z)
• Zero-Shot Cross-lingual Semantic Parsing [56.95036511882921]
  We study cross-lingual semantic parsing as a zero-shot problem without parallel data for 7 test languages. We propose a multi-task encoder-decoder model to transfer parsing knowledge to additional languages using only English-Logical form paired data. Our system frames zero-shot parsing as a latent-space alignment problem and finds that pre-trained models can be improved to generate logical forms with minimal cross-lingual transfer penalty.
  arXiv  Detail & Related papers  (2021-04-15T16:08:43Z)
• Word Alignment by Fine-tuning Embeddings on Parallel Corpora [96.28608163701055]
  Word alignment over parallel corpora has a wide variety of applications, including learning translation lexicons, cross-lingual transfer of language processing tools, and automatic evaluation or analysis of translation outputs. Recently, other work has demonstrated that pre-trained contextualized word embeddings derived from multilingually trained language models (LMs) prove an attractive alternative, achieving competitive results on the word alignment task even in the absence of explicit training on parallel data. In this paper, we examine methods to marry the two approaches: leveraging pre-trained LMs but fine-tuning them on parallel text with objectives designed to improve alignment quality, and proposing
  arXiv  Detail & Related papers  (2021-01-20T17:54:47Z)
• Subword Sampling for Low Resource Word Alignment [4.663577299263155]
  We propose subword sampling-based alignment of text units. We show that the subword sampling method consistently outperforms word-level alignment on six language pairs.
  arXiv  Detail & Related papers  (2020-12-21T19:47:04Z)
• Learning Contextualised Cross-lingual Word Embeddings and Alignments for Extremely Low-Resource Languages Using Parallel Corpora [63.5286019659504]
  We propose a new approach for learning contextualised cross-lingual word embeddings based on a small parallel corpus. Our method obtains word embeddings via an LSTM encoder-decoder model that simultaneously translates and reconstructs an input sentence.
  arXiv  Detail & Related papers  (2020-10-27T22:24:01Z)
• Cross-lingual Alignment Methods for Multilingual BERT: A Comparative Study [2.101267270902429]
  We analyse how different forms of cross-lingual supervision and various alignment methods influence the transfer capability of mBERT in zero-shot setting. We find that supervision from parallel corpus is generally superior to dictionary alignments.
  arXiv  Detail & Related papers  (2020-09-29T20:56:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.