Semantic Alignment across Ancient Egyptian Language Stages via Normalization-Aware Multitask Learning

• URL: http://arxiv.org/abs/2603.24258v1
• Date: Wed, 25 Mar 2026 12:52:38 GMT
• Title: Semantic Alignment across Ancient Egyptian Language Stages via Normalization-Aware Multitask Learning
• Abstract summary: We study word-level semantic alignment across four historical stages of Ancient Egyptian.<n>We train a compact encoder-decoder model with a shared byte-level tokenizer on all four stages.<n>We integrate Latin transliteration and IPA reconstruction as auxiliary views.
• Score: 5.63985701809237
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study word-level semantic alignment across four historical stages of Ancient Egyptian. These stages differ in script and orthography, and parallel data are scarce. We jointly train a compact encoder-decoder model with a shared byte-level tokenizer on all four stages, combining masked language modeling (MLM), translation language modeling (TLM), sequence-to-sequence translation, and part-of-speech tagging under a task-aware loss with fixed weights and uncertainty-based scaling. To reduce surface divergence we add Latin transliteration and IPA reconstruction as auxiliary views. We integrate these views through KL-based consistency and through embedding-level fusion. We evaluate alignment quality using pairwise metrics, specifically ROC-AUC and triplet accuracy, on curated Egyptian-English and intra-Egyptian cognate datasets. Translation yields the strongest gains. IPA with KL consistency improves cross-branch alignment, while early fusion demonstrates limited efficacy. Although the overall alignment remains limited, the findings provide a reproducible baseline and practical guidance for modeling historical languages under real constraints. They also show how normalization and task design shape what counts as alignment in typologically distant settings.

Related papers

• Discovering Cross-Language Reasoning Invariance in LLMs with Geometry-Invariant Sparse Autoencoders [0.7555681563742543]
  We study whether multilingual language models rely on shared features or language-specific computations that only produce similar outputs.<n>We find that cross-language feature sharing is model- and architecture-dependent in this sample and appears at different depths in different models.
  arXiv  Detail & Related papers  (2026-08-24T20:17:55Z)
• DATASHI: A Parallel English-Tashlhiyt Corpus for Orthography Normalization and Low-Resource Language Processing [0.0]
  DATASHI is a new parallel English-Tashlhiyt corpus that fills a critical gap in computational resources for Amazigh languages.<n>It contains 5,000 sentence pairs, including a 1,500-sentence subset with expert-standardized and non-standard user-generated versions.
  arXiv  Detail & Related papers  (2026-03-23T04:46:59Z)
• TransAlign: Machine Translation Encoders are Strong Word Aligners, Too [5.078113219758536]
  We propose TransAlign, a novel word aligner that utilizes the encoder of a massively multilingual machine translation model.<n>We show that TransAlign achieves strong WA performance and substantially outperforms popular WA and state-of-the-art non-WA-based label projection methods.
  arXiv  Detail & Related papers  (2025-10-31T10:14:51Z)
• Linguistics-Vision Monotonic Consistent Network for Sign Language Production [45.12628941399177]
  Sign Language Production (SLP) aims to generate sign videos corresponding to spoken language sentences.<n>Due to the cross-modal semantic gap, the SLP suffers huge challenges in linguistics-vision consistency.<n>We propose a Transformer-based Linguistics-Vision Monotonic Consistent Network (LVMCN) for SLP.
  arXiv  Detail & Related papers  (2024-12-22T09:28:06Z)
• Language Models as Hierarchy Encoders [22.03504018330068]
  We introduce a novel approach to re-train transformer encoder-based LMs as Hierarchy Transformer encoders (HiTs) Our method situates the output embedding space of pre-trained LMs within a Poincar'e ball with a curvature that adapts to the embedding dimension. We evaluate HiTs against pre-trained LMs, standard fine-tuned LMs, and several hyperbolic embedding baselines.
  arXiv  Detail & Related papers  (2024-01-21T02:29:12Z)
• Salute the Classic: Revisiting Challenges of Machine Translation in the Age of Large Language Models [91.6543868677356]
  The evolution of Neural Machine Translation has been influenced by six core challenges. These challenges include domain mismatch, amount of parallel data, rare word prediction, translation of long sentences, attention model as word alignment, and sub-optimal beam search. This study revisits these challenges, offering insights into their ongoing relevance in the context of advanced Large Language Models.
  arXiv  Detail & Related papers  (2024-01-16T13:30:09Z)
• Dual-Alignment Pre-training for Cross-lingual Sentence Embedding [79.98111074307657]
  We propose a dual-alignment pre-training (DAP) framework for cross-lingual sentence embedding. We introduce a novel representation translation learning (RTL) task, where the model learns to use one-side contextualized token representation to reconstruct its translation counterpart. Our approach can significantly improve sentence embedding.
  arXiv  Detail & Related papers  (2023-05-16T03:53:30Z)
• VECO 2.0: Cross-lingual Language Model Pre-training with Multi-granularity Contrastive Learning [56.47303426167584]
  We propose a cross-lingual pre-trained model VECO2.0 based on contrastive learning with multi-granularity alignments. Specifically, the sequence-to-sequence alignment is induced to maximize the similarity of the parallel pairs and minimize the non-parallel pairs. token-to-token alignment is integrated to bridge the gap between synonymous tokens excavated via the thesaurus dictionary from the other unpaired tokens in a bilingual instance.
  arXiv  Detail & Related papers  (2023-04-17T12:23:41Z)
• Neural Machine Translation with Contrastive Translation Memories [71.86990102704311]
  Retrieval-augmented Neural Machine Translation models have been successful in many translation scenarios. We propose a new retrieval-augmented NMT to model contrastively retrieved translation memories that are holistically similar to the source sentence. In training phase, a Multi-TM contrastive learning objective is introduced to learn salient feature of each TM with respect to target sentence.
  arXiv  Detail & Related papers  (2022-12-06T17:10:17Z)
• Improving Multilingual Translation by Representation and Gradient Regularization [82.42760103045083]
  We propose a joint approach to regularize NMT models at both representation-level and gradient-level. Our results demonstrate that our approach is highly effective in both reducing off-target translation occurrences and improving zero-shot translation performance.
  arXiv  Detail & Related papers  (2021-09-10T10:52:21Z)
• Exploring Unsupervised Pretraining Objectives for Machine Translation [99.5441395624651]
  Unsupervised cross-lingual pretraining has achieved strong results in neural machine translation (NMT) Most approaches adapt masked-language modeling (MLM) to sequence-to-sequence architectures, by masking parts of the input and reconstructing them in the decoder. We compare masking with alternative objectives that produce inputs resembling real (full) sentences, by reordering and replacing words based on their context.
  arXiv  Detail & Related papers  (2021-06-10T10:18:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.