Scaling Unsupervised Word Alignment to Documents via Structural Constraints

• URL: http://arxiv.org/abs/2608.21023v1
• Date: Fri, 21 Aug 2026 12:13:21 GMT
• Title: Scaling Unsupervised Word Alignment to Documents via Structural Constraints
• Abstract summary: We introduce CTFAlign, a lightweight, training-free approach for document-level word alignment.<n>We also introduce MDPAlign, a simpler alternative that constrains alignments by position with a main diagonal prior.<n>Averaged over three models, CTFAlign reduces word alignment error rate from 0.412 to 0.326.
• Score: 38.797311337915175
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Word alignment has traditionally been studied between sentences, but many cross-lingual tasks increasingly require correspondences across full documents. While recent multilingual embedding models can encode long inputs, we show that applying algorithms designed for sentences directly to documents leads to performance degradation. To address this, we introduce CTFAlign, a lightweight, training-free approach for document-level word alignment. CTFAlign applies a coarse-to-fine refinement strategy that restricts the alignment search space to semantically similar regions. Additionally, we introduce MDPAlign, a simpler alternative that constrains alignments by position with a main diagonal prior. Both approaches operate directly on full documents without relying on sentence segmentation or sentence alignment. We evaluate these methods across six language pairs varying in typological distance, resourcedness, and document length. Averaged over three models, CTFAlign reduces word alignment error rate from 0.412 to 0.326. These gains transfer downstream, leading to improvements in document-level translation coverage evaluation and recognition of semantic differences. We release CTFAlign as a Python package and make the code and data to reproduce our experiments publicly available.

Related papers

• OmniAlign: A Unified Multilingual Aligner for Word and Sentence Alignment [10.629976569255229]
  We present OmniAlign, a unified multilingual aligner that supports both word-level and sentence-level alignment.<n>Built on an encoder-only backbone with strong long-context modeling, OmniAlign induces word alignments from contextualized token similarity matrices.<n>Experiments show that OmniAlign achieves highly competitive performance on both word- and sentence-alignment benchmarks.
  arXiv  Detail & Related papers  (2026-08-19T03:02:43Z)
• Pralekha: Cross-Lingual Document Alignment for Indic Languages [24.881962056284692]
  Document Alignment Coefficient (DAC) is a novel metric for fine-grained document alignment.<n>Unlike pooling-based methods, DAC aligns documents by matching smaller chunks and computes similarity as the ratio of aligned chunks to the average number of chunks in a pair.
  arXiv  Detail & Related papers  (2024-11-28T12:17:24Z)
• Contextual Document Embeddings [77.22328616983417]
  We propose two complementary methods for contextualized document embeddings. First, an alternative contrastive learning objective that explicitly incorporates the document neighbors into the intra-batch contextual loss. Second, a new contextual architecture that explicitly encodes neighbor document information into the encoded representation.
  arXiv  Detail & Related papers  (2024-10-03T14:33:34Z)
• In-context Pretraining: Language Modeling Beyond Document Boundaries [137.53145699439898]
  In-Context Pretraining is a new approach where language models are pretrained on a sequence of related documents. We introduce approximate algorithms for finding related documents with efficient nearest neighbor search. We see notable improvements in tasks that require more complex contextual reasoning.
  arXiv  Detail & Related papers  (2023-10-16T17:57:12Z)
• Towards Unsupervised Recognition of Token-level Semantic Differences in Related Documents [61.63208012250885]
  We formulate recognizing semantic differences as a token-level regression task. We study three unsupervised approaches that rely on a masked language model. Our results show that an approach based on word alignment and sentence-level contrastive learning has a robust correlation to gold labels.
  arXiv  Detail & Related papers  (2023-05-22T17:58:04Z)
• Are the Best Multilingual Document Embeddings simply Based on Sentence Embeddings? [18.968571816913208]
  We provide a systematic comparison of methods to produce document-level representations from sentences based on LASER, LaBSE, and Sentence BERT pre-trained multilingual models. We show that a clever combination of sentence embeddings is usually better than encoding the full document as a single unit.
  arXiv  Detail & Related papers  (2023-04-28T12:11:21Z)
• Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment [49.45399359826453]
  Cross-lingual language models are typically pretrained with language modeling on multilingual text or parallel sentences. We introduce denoising word alignment as a new cross-lingual pre-training task. Experimental results show that our method improves cross-lingual transferability on various datasets.
  arXiv  Detail & Related papers  (2021-06-11T13:36:01Z)
• Word Alignment by Fine-tuning Embeddings on Parallel Corpora [96.28608163701055]
  Word alignment over parallel corpora has a wide variety of applications, including learning translation lexicons, cross-lingual transfer of language processing tools, and automatic evaluation or analysis of translation outputs. Recently, other work has demonstrated that pre-trained contextualized word embeddings derived from multilingually trained language models (LMs) prove an attractive alternative, achieving competitive results on the word alignment task even in the absence of explicit training on parallel data. In this paper, we examine methods to marry the two approaches: leveraging pre-trained LMs but fine-tuning them on parallel text with objectives designed to improve alignment quality, and proposing
  arXiv  Detail & Related papers  (2021-01-20T17:54:47Z)
• Learning aligned embeddings for semi-supervised word translation using Maximum Mean Discrepancy [3.299672391663527]
  We propose an end-to-end approach for word embedding alignment that does not require known word pairs. Our method learns embeddings that are aligned during sentence translation training using a localized Maximum Mean Discrepancy (MMD) constraint. We show that our method not only out-performs unsupervised methods, but also supervised methods that train on known word translations.
  arXiv  Detail & Related papers  (2020-06-20T13:57:55Z)
• Massively Multilingual Document Alignment with Cross-lingual Sentence-Mover's Distance [8.395430195053061]
  Document alignment aims to identify pairs of documents in two distinct languages that are of comparable content or translations of each other. We develop an unsupervised scoring function that leverages cross-lingual sentence embeddings to compute the semantic distance between documents in different languages. These semantic distances are then used to guide a document alignment algorithm to properly pair cross-lingual web documents across a variety of low, mid, and high-resource language pairs.
  arXiv  Detail & Related papers  (2020-01-31T05:14:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.