Morphemes Without Borders: Evaluating Root-Pattern Morphology in Arabic Tokenizers and LLMs

• URL: http://arxiv.org/abs/2603.15773v1
• Date: Mon, 16 Mar 2026 18:05:01 GMT
• Title: Morphemes Without Borders: Evaluating Root-Pattern Morphology in Arabic Tokenizers and LLMs
• Abstract summary: This work investigates how effectively large language models (LLMs) represent and generate Arabic root-pattern morphology.<n>Our study begins with an evaluation of morphological fidelity across Arabic and multilingual tokenizers against gold-standard segmentation.<n>Our findings reveal that tokenizer morphological alignment is not necessary nor sufficient for morphological generation, which questions the role of morphological tokenization in downstream performance.
• Score: 6.520393736113561
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work investigates how effectively large language models (LLMs) and their tokenization schemes represent and generate Arabic root-pattern morphology, probing whether they capture genuine morphological structure or rely on surface memorization. Arabic morphological system provides a rich testbed for analyzing how LLMs handle complex, non-concatenative forms and how tokenization choices influence this process. Our study begins with an evaluation of morphological fidelity across Arabic and multilingual tokenizers against gold-standard segmentation, followed by an analysis of LLM performance in productive root-pattern generation using a newly developed test set. Our findings across seven Arabic-centric and multilingual LLMs and their respective tokenizers reveal that tokenizer morphological alignment is not necessary nor sufficient for morphological generation, which questions the role of morphological tokenization in downstream performance.

Related papers

• YallaMorph: A Benchmark for Evaluating Arabic Morphological Generation in Large Language Models [12.343629326707733]
  We introduce YallaMorph, a large-scale benchmark for Arabic morphological generation.<n>We evaluate multilingual and Arabic-oriented LLMs under diacritized and undiacritized settings over 600K benchmark entries.<n>Results show that Arabic morphological generation remains difficult, especially for cliticized, unseen, and morphologically rare forms.
  arXiv  Detail & Related papers  (2026-09-09T13:31:56Z)
• Arabic Morphosyntactic Tagging and Dependency Parsing with Large Language Models [13.143003615122245]
  Large language models (LLMs) perform strongly on many NLP tasks, but their ability to produce explicit linguistic structure remains unclear.<n>We evaluate instruction-tuned LLMs on two structured prediction tasks for Standard Arabic: morphosyntactic tagging and labeled dependency parsing.
  arXiv  Detail & Related papers  (2026-03-17T16:06:29Z)
• IMPACT: Inflectional Morphology Probes Across Complex Typologies [0.0]
  IMPACT is a synthetically generated evaluation framework focused on inflectional morphology.<n>It is designed to evaluate performance across five morphologically rich languages: Arabic, Russian, Finnish, Turkish, and Hebrew.<n>We assess eight multilingual LLMs that, despite strong English performance, struggle with other languages and uncommon morphological patterns.
  arXiv  Detail & Related papers  (2025-06-30T14:58:23Z)
• Unnatural Languages Are Not Bugs but Features for LLMs [92.8332103170009]
  Large Language Models (LLMs) have been observed to process non-human-readable text sequences, such as jailbreak prompts.<n>We present a systematic investigation challenging this perception, demonstrating that unnatural languages contain latent features usable by models.
  arXiv  Detail & Related papers  (2025-03-02T12:10:17Z)
• Evaluating Morphological Compositional Generalization in Large Language Models [17.507983593566223]
  We investigate the morphological generalization abilities of large language models (LLMs) through the lens of compositionality.<n>We focus on agglutinative languages such as Turkish and Finnish.<n>Our analysis shows that LLMs struggle with morphological compositional generalization particularly when applied to novel word roots.<n>While models can identify individual morphological combinations better than chance, their performance lacks systematicity, leading to significant accuracy gaps compared to humans.
  arXiv  Detail & Related papers  (2024-10-16T15:17:20Z)
• A Morphology-Based Investigation of Positional Encodings [46.667985003225496]
  Morphology and word order are closely linked, with the latter incorporated into transformer-based models through positional encodings. This prompts a fundamental inquiry: Is there a correlation between the morphological complexity of a language and the utilization of positional encoding in pre-trained language models? In pursuit of an answer, we present the first study addressing this question, encompassing 22 languages and 5 downstream tasks.
  arXiv  Detail & Related papers  (2024-04-06T07:10:47Z)
• How Proficient Are Large Language Models in Formal Languages? An In-Depth Insight for Knowledge Base Question Answering [52.86931192259096]
  Knowledge Base Question Answering (KBQA) aims to answer natural language questions based on facts in knowledge bases. Recent works leverage the capabilities of large language models (LLMs) for logical form generation to improve performance.
  arXiv  Detail & Related papers  (2024-01-11T09:27:50Z)
• Explicit Morphological Knowledge Improves Pre-training of Language Models for Hebrew [19.4968960182412]
  We investigate the hypothesis that incorporating explicit morphological knowledge in the pre-training phase can improve the performance of PLMs for morphologically rich languages. We propose various morphologically driven tokenization methods enabling the model to leverage morphological cues beyond raw text. Our experiments show that morphologically driven tokenization demonstrates improved results compared to a standard language-agnostic tokenization.
  arXiv  Detail & Related papers  (2023-11-01T17:02:49Z)
• Evaluating the Morphosyntactic Well-formedness of Generated Texts [88.20502652494521]
  We propose L'AMBRE -- a metric to evaluate the morphosyntactic well-formedness of text. We show the effectiveness of our metric on the task of machine translation through a diachronic study of systems translating into morphologically-rich languages.
  arXiv  Detail & Related papers  (2021-03-30T18:02:58Z)
• Evaluating Transformer-Based Multilingual Text Classification [55.53547556060537]
  We argue that NLP tools perform unequally across languages with different syntactic and morphological structures. We calculate word order and morphological similarity indices to aid our empirical study.
  arXiv  Detail & Related papers  (2020-04-29T03:34:53Z)
• A Simple Joint Model for Improved Contextual Neural Lemmatization [60.802451210656805]
  We present a simple joint neural model for lemmatization and morphological tagging that achieves state-of-the-art results on 20 languages. Our paper describes the model in addition to training and decoding procedures.
  arXiv  Detail & Related papers  (2019-04-04T02:03:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.