OmniAlign: A Unified Multilingual Aligner for Word and Sentence Alignment

• URL: http://arxiv.org/abs/2608.18474v1
• Date: Wed, 19 Aug 2026 03:02:43 GMT
• Title: OmniAlign: A Unified Multilingual Aligner for Word and Sentence Alignment
• Abstract summary: We present OmniAlign, a unified multilingual aligner that supports both word-level and sentence-level alignment.<n>Built on an encoder-only backbone with strong long-context modeling, OmniAlign induces word alignments from contextualized token similarity matrices.<n>Experiments show that OmniAlign achieves highly competitive performance on both word- and sentence-alignment benchmarks.
• Score: 10.629976569255229
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Cross-lingual sequence alignment is fundamental for building and exploiting parallel corpora, spanning mappings from documents and sentences down to words and subwords. Existing tools, however, typically specialize in a single granularity, so practitioners often need separate systems for word- and sentence-level alignment---especially in multilingual and long-text settings. We present OmniAlign, a unified multilingual aligner that supports both word-level and sentence-level alignment with a single lightweight model. Built on an encoder-only backbone with strong long-context modeling, OmniAlign induces word alignments from contextualized token similarity matrices, and obtains document-level $m$--$n$ sentence alignments via sentence embeddings combined with dynamic programming. To balance fine-grained alignment accuracy and sentence-representation quality, we use a four-stage training pipeline: alignment-oriented continued pre-training, self-supervised learning, supervised fine-tuning on human annotations, and sentence-embedding distillation from a strong multilingual teacher. Experiments show that OmniAlign achieves highly competitive performance on both word- and sentence-alignment benchmarks and generalizes well to unseen language pairs. Surprisingly, later-stage supervised fine-tuning on short texts further improves alignment quality while retaining the long-context understanding acquired in earlier training, keeping the model robust on long-text word alignment. \normalsize {\color{blue}\textbf{Code}: https://github.com/MilkDargon/OmniAlign}\par {\color{blue}\textbf{Model}: https://huggingface.co/WPS-Qingqiu/OmniAlign}

Related papers

• Scaling Unsupervised Word Alignment to Documents via Structural Constraints [38.797311337915175]
  We introduce CTFAlign, a lightweight, training-free approach for document-level word alignment.<n>We also introduce MDPAlign, a simpler alternative that constrains alignments by position with a main diagonal prior.<n>Averaged over three models, CTFAlign reduces word alignment error rate from 0.412 to 0.326.
  arXiv  Detail & Related papers  (2026-08-21T12:13:21Z)
• Simultaneous Speech-to-Speech Translation Without Aligned Data [52.467808474293605]
  Simultaneous speech translation requires translating source speech into a target language in real-time.<n>We propose Hibiki-Zero, which eliminates the need for word-level alignments entirely.<n>Hibiki-Zero achieves state-of-the-art performance in translation accuracy, latency, voice transfer, and naturalness across five X-to-English tasks.
  arXiv  Detail & Related papers  (2026-02-11T17:41:01Z)
• Third-Party Aligner for Neural Word Alignments [18.745852103348845]
  We propose to use word alignments generated by a third-party word aligner to supervise the neural word alignment training. Experiments show that our approach can surprisingly do self-correction over the third-party supervision. We achieve state-of-the-art word alignment performances, with averagely more than two points lower alignment error rates than the best third-party aligner.
  arXiv  Detail & Related papers  (2022-11-08T12:30:08Z)
• Combining Static and Contextualised Multilingual Embeddings [70.12036693730919]
  We combine the strengths of static and contextual models to improve multilingual representations. We extract static embeddings for 40 languages from XLM-R, validate those embeddings with cross-lingual word retrieval, and then align them using VecMap. This results in high-quality, highly multilingual static embeddings.
  arXiv  Detail & Related papers  (2022-03-17T13:53:46Z)
• Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment [49.45399359826453]
  Cross-lingual language models are typically pretrained with language modeling on multilingual text or parallel sentences. We introduce denoising word alignment as a new cross-lingual pre-training task. Experimental results show that our method improves cross-lingual transferability on various datasets.
  arXiv  Detail & Related papers  (2021-06-11T13:36:01Z)
• Word Alignment by Fine-tuning Embeddings on Parallel Corpora [96.28608163701055]
  Word alignment over parallel corpora has a wide variety of applications, including learning translation lexicons, cross-lingual transfer of language processing tools, and automatic evaluation or analysis of translation outputs. Recently, other work has demonstrated that pre-trained contextualized word embeddings derived from multilingually trained language models (LMs) prove an attractive alternative, achieving competitive results on the word alignment task even in the absence of explicit training on parallel data. In this paper, we examine methods to marry the two approaches: leveraging pre-trained LMs but fine-tuning them on parallel text with objectives designed to improve alignment quality, and proposing
  arXiv  Detail & Related papers  (2021-01-20T17:54:47Z)
• Subword Sampling for Low Resource Word Alignment [4.663577299263155]
  We propose subword sampling-based alignment of text units. We show that the subword sampling method consistently outperforms word-level alignment on six language pairs.
  arXiv  Detail & Related papers  (2020-12-21T19:47:04Z)
• Learning Contextualised Cross-lingual Word Embeddings and Alignments for Extremely Low-Resource Languages Using Parallel Corpora [63.5286019659504]
  We propose a new approach for learning contextualised cross-lingual word embeddings based on a small parallel corpus. Our method obtains word embeddings via an LSTM encoder-decoder model that simultaneously translates and reconstructs an input sentence.
  arXiv  Detail & Related papers  (2020-10-27T22:24:01Z)
• SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings [3.8424737607413153]
  We propose word alignment methods that require no parallel data. Key idea is to leverage multilingual word embeddings, both static and contextualized, for word alignment. We find that alignments created from embeddings are superior for two language pairs compared to those produced by traditional statistical methods.
  arXiv  Detail & Related papers  (2020-04-18T23:10:36Z)
• Multilingual Alignment of Contextual Word Representations [49.42244463346612]
  BERT exhibits significantly improved zero-shot performance on XNLI compared to the base model. We introduce a contextual version of word retrieval and show that it correlates well with downstream zero-shot transfer. These results support contextual alignment as a useful concept for understanding large multilingual pre-trained models.
  arXiv  Detail & Related papers  (2020-02-10T03:27:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.