CWoMP: Morpheme Representation Learning for Interlinear Glossing

• URL: http://arxiv.org/abs/2603.18184v1
• Date: Wed, 18 Mar 2026 18:29:18 GMT
• Title: CWoMP: Morpheme Representation Learning for Interlinear Glossing
• Abstract summary: We propose CWoMP (Contrastive Word-Morpheme Pretraining), which treats morphemes as atomic form-meaning units with learned representations.<n>We evaluate on diverse low-resource languages, showing that CWoMP outperforms existing methods while being significantly more efficient.
• Score: 16.271227825969433
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Interlinear glossed text (IGT) is a standard notation for language documentation which is linguistically rich but laborious to produce manually. Recent automated IGT methods treat glosses as character sequences, neglecting their compositional structure. We propose CWoMP (Contrastive Word-Morpheme Pretraining), which instead treats morphemes as atomic form-meaning units with learned representations. A contrastively trained encoder aligns words-in-context with their constituent morphemes in a shared embedding space; an autoregressive decoder then generates the morpheme sequence by retrieving entries from a mutable lexicon of these embeddings. Predictions are interpretable--grounded in lexicon entries--and users can improve results at inference time by expanding the lexicon without retraining. We evaluate on diverse low-resource languages, showing that CWoMP outperforms existing methods while being significantly more efficient, with particularly strong gains in extremely low-resource settings.

Related papers

• Weave of Formal Thought [51.56484100374058]
  We introduce Weave of Formal Thought (WoFT), a paradigm uniting rigorous syntactic validation with learned structural representations.<n>We present a latent-variable fine-tuning method training the language model to interleave non-terminal grammar symbols directly into generation.<n>For Python, fine-tuning StarCoder2-3B with our RWS objective reduces per-token cross-entropy by 14.3% relative to a text-only SFT baseline.
  arXiv  Detail & Related papers  (2026-06-24T15:58:11Z)
• GlossAssist -- A Tool to Simplify Corpus Creation and Study the Effect of NLP Models in Low-Resource Documentation Settings [10.11584948505218]
  We present GlossAssist, a glossing tool built around the retrieval-based architecture of CWoMP.<n>Our system treats each correction by an annotator as part of an active learning setting, which expands the lexicon and improves future predictions without having to retrain the model.
  arXiv  Detail & Related papers  (2026-06-03T02:29:30Z)
• LGSE: Lexically Grounded Subword Embedding Initialization for Low-Resource Language Adaptation [7.623227616015147]
  We propose the Lexically Grounded Subword Embedding Initialization framework, which introduces morphologically informed segmentation for initializing embeddings of novel tokens.<n>Instead of using random vectors or arbitrary subwords, LGSE decomposes words into their constituent morphemes and constructs semantically coherent embeddings.<n>We evaluate LGSE on three NLP tasks: Question Answering, Named Entity Recognition, and Text Classification, in two morphologically rich, low-resource languages.
  arXiv  Detail & Related papers  (2026-03-23T23:07:16Z)
• MoVoC: Morphology-Aware Subword Construction for Geez Script Languages [7.7761618950496265]
  Subword-based tokenization methods often fail to preserve morphological boundaries.<n>We present MoVoC (Morpheme-aware Subword Vocabulary Construction) and train MoVoC-Tok, a tokenizer that integrates supervised morphological analysis into the subword vocabulary.
  arXiv  Detail & Related papers  (2025-09-10T17:45:10Z)
• Resource-Efficient Adaptation of Large Language Models for Text Embeddings via Prompt Engineering and Contrastive Fine-tuning [3.9914181590063884]
  Large Language Models (LLMs) have become a cornerstone in Natural Language Processing (NLP)<n>We explore several adaptation strategies for pre-trained, decoder-only LLMs.
  arXiv  Detail & Related papers  (2025-07-30T14:49:30Z)
• Contextualized Automatic Speech Recognition with Dynamic Vocabulary [41.892863381787684]
  This paper proposes a dynamic vocabulary where bias tokens can be added during inference. Each entry in a bias list is represented as a single token, unlike a sequence of existing subword tokens. Experimental results demonstrate that the proposed method improves the bias phrase WER on English and Japanese datasets by 3.1 -- 4.9 points.
  arXiv  Detail & Related papers  (2024-05-22T05:03:39Z)
• A General and Flexible Multi-concept Parsing Framework for Multilingual Semantic Matching [60.51839859852572]
  We propose to resolve the text into multi concepts for multilingual semantic matching to liberate the model from the reliance on NER models. We conduct comprehensive experiments on English datasets QQP and MRPC, and Chinese dataset Medical-SM.
  arXiv  Detail & Related papers  (2024-03-05T13:55:16Z)
• Contrastive Instruction Tuning [61.97704869248903]
  We propose Contrastive Instruction Tuning to maximize the similarity between semantically equivalent instruction-instance pairs. Experiments on the PromptBench benchmark show that CoIN consistently improves LLMs' robustness to unseen instructions with variations across character, word, sentence, and semantic levels by an average of +2.5% in accuracy.
  arXiv  Detail & Related papers  (2024-02-17T00:09:32Z)
• Nugget: Neural Agglomerative Embeddings of Text [49.233014020133034]
  We propose a solution called Nugget, which encodes language into a representation based on a subset of input tokens. These nuggets are learned through tasks like autoencoding and machine translation, and intuitively segment language into meaningful units.
  arXiv  Detail & Related papers  (2023-10-03T01:47:49Z)
• More Romanian word embeddings from the RETEROM project [0.0]
  "word embeddings" are automatically learned vector representations of words. We plan to develop an openaccess large library of ready-to-use word embeddings sets.
  arXiv  Detail & Related papers  (2021-11-21T06:05:12Z)
• Charformer: Fast Character Transformers via Gradient-based Subword Tokenization [50.16128796194463]
  We propose a new model inductive bias that learns a subword tokenization end-to-end as part of the model. We introduce a soft gradient-based subword tokenization module (GBST) that automatically learns latent subword representations from characters. We additionally introduce Charformer, a deep Transformer model that integrates GBST and operates on the byte level.
  arXiv  Detail & Related papers  (2021-06-23T22:24:14Z)
• POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training [93.79766670391618]
  We present POINTER, a novel insertion-based approach for hard-constrained text generation. The proposed method operates by progressively inserting new tokens between existing tokens in a parallel manner. The resulting coarse-to-fine hierarchy makes the generation process intuitive and interpretable.
  arXiv  Detail & Related papers  (2020-05-01T18:11:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.