PACUTE: Phonology-, Affix-, and Character-level Understanding of Tokens for Filipino

• URL: http://arxiv.org/abs/2606.15144v1
• Date: Sat, 13 Jun 2026 06:12:56 GMT
• Title: PACUTE: Phonology-, Affix-, and Character-level Understanding of Tokens for Filipino
• Abstract summary: PACUTE is a diagnostic benchmark of 4,600 tasks designed to evaluate morphological understanding in Filipino.<n>We find that open-weight models perform near chance on morpheme decomposition regardless of scale.<n>These results identify productive morphological composition, rather than character access alone, as the persistent bottleneck for Filipino word-structure understanding.
• Score: 1.6481883307499954
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Large language models (LLMs) process text as sequences of subword tokens, which can obscure the character-level and morphological structure that underlies word formation. This limitation is most acute for languages with non-concatenative morphology, where standard tokenizers systematically misalign token boundaries with morpheme boundaries. We introduce PACUTE, a diagnostic benchmark of 4,600 tasks designed to evaluate morphological understanding in Filipino, a language characterized by productive infixation, reduplication, and diacritic-driven lexical distinctions that are typically absent from written text. PACUTE includes a hierarchical diagnostic framework of six compositional levels that localizes where morphological understanding breaks down. Evaluating open-weight LLMs and frontier commercial models, we find that open-weight models perform near chance on morpheme decomposition regardless of scale. Frontier models perform much better, often recovering individual affixes under contains-match scoring, but remain far below their character-level ceilings on compositional tasks of morpheme transformations and syllabification. These results identify productive morphological composition, rather than character access alone, as the persistent bottleneck for Filipino word-structure understanding.

Related papers

• Morphemes Without Borders: Evaluating Root-Pattern Morphology in Arabic Tokenizers and LLMs [6.520393736113561]
  This work investigates how effectively large language models (LLMs) represent and generate Arabic root-pattern morphology.<n>Our study begins with an evaluation of morphological fidelity across Arabic and multilingual tokenizers against gold-standard segmentation.<n>Our findings reveal that tokenizer morphological alignment is not necessary nor sufficient for morphological generation, which questions the role of morphological tokenization in downstream performance.
  arXiv  Detail & Related papers  (2026-03-16T18:05:01Z)
• chDzDT: Word-level morphology-aware language model for Algerian social media text [0.0]
  chDzDT is a character-level pre-trained language model tailored for Algerian morphology.<n>It is trained on isolated words without depending on token boundaries or standardized orthography.<n>It covers multiple scripts and linguistic varieties, resulting in a substantial pre-training workload.
  arXiv  Detail & Related papers  (2025-09-01T21:09:55Z)
• Unsupervised Morphological Tree Tokenizer [36.584680344291556]
  We introduce morphological structure guidance to tokenization and propose a deep model to induce character-level structures of words.<n>Our method is capable of inducing character-level structures that align with morphological rules without annotated training data.<n> Empirical results indicate that the proposed method effectively retains complete morphemes and outperforms widely adopted methods such as BPE and WordPiece.
  arXiv  Detail & Related papers  (2024-06-21T15:35:49Z)
• Multilingual Extraction and Categorization of Lexical Collocations with Graph-aware Transformers [86.64972552583941]
  We put forward a sequence tagging BERT-based model enhanced with a graph-aware transformer architecture, which we evaluate on the task of collocation recognition in context. Our results suggest that explicitly encoding syntactic dependencies in the model architecture is helpful, and provide insights on differences in collocation typification in English, Spanish and French.
  arXiv  Detail & Related papers  (2022-05-23T16:47:37Z)
• UniMorph 4.0: Universal Morphology [104.69846084893298]
  This paper presents the expansions and improvements made on several fronts over the last couple of years. Collaborative efforts by numerous linguists have added 67 new languages, including 30 endangered languages. In light of the last UniMorph release, we also augmented the database with morpheme segmentation for 16 languages.
  arXiv  Detail & Related papers  (2022-05-07T09:19:02Z)
• Morphology Without Borders: Clause-Level Morphological Annotation [8.559428282730021]
  We propose to view morphology as a clause-level phenomenon, rather than word-level. We deliver a novel dataset for clause-level morphology covering 4 typologically-different languages: English, German, Turkish and Hebrew. Our experiments show that the clause-level tasks are substantially harder than the respective word-level tasks, while having comparable complexity across languages.
  arXiv  Detail & Related papers  (2022-02-25T17:20:28Z)
• A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space [61.18554842370824]
  In cross-lingual language models, representations for many different languages live in the same space. We compute a task-based measure of cross-lingual alignment in the form of bitext retrieval performance. We examine a range of linguistic, quasi-linguistic, and training-related features as potential predictors of these alignment metrics.
  arXiv  Detail & Related papers  (2021-09-13T21:05:37Z)
• How Suitable Are Subword Segmentation Strategies for Translating Non-Concatenative Morphology? [26.71325671956197]
  We design a test suite to evaluate segmentation strategies on different types of morphological phenomena. We find that learning to analyse and generate morphologically complex surface representations is still challenging.
  arXiv  Detail & Related papers  (2021-09-02T17:23:21Z)
• Evaluating the Morphosyntactic Well-formedness of Generated Texts [88.20502652494521]
  We propose L'AMBRE -- a metric to evaluate the morphosyntactic well-formedness of text. We show the effectiveness of our metric on the task of machine translation through a diachronic study of systems translating into morphologically-rich languages.
  arXiv  Detail & Related papers  (2021-03-30T18:02:58Z)
• Evaluating Transformer-Based Multilingual Text Classification [55.53547556060537]
  We argue that NLP tools perform unequally across languages with different syntactic and morphological structures. We calculate word order and morphological similarity indices to aid our empirical study.
  arXiv  Detail & Related papers  (2020-04-29T03:34:53Z)
• A Simple Joint Model for Improved Contextual Neural Lemmatization [60.802451210656805]
  We present a simple joint neural model for lemmatization and morphological tagging that achieves state-of-the-art results on 20 languages. Our paper describes the model in addition to training and decoding procedures.
  arXiv  Detail & Related papers  (2019-04-04T02:03:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.