UzMorphAnalyser: A Morphological Analysis Model for the Uzbek Language Using Inflectional Endings

• URL: http://arxiv.org/abs/2405.14179v2
• Date: Wed, 12 Jun 2024 09:16:46 GMT
• Title: UzMorphAnalyser: A Morphological Analysis Model for the Uzbek Language Using Inflectional Endings
• Authors: Ulugbek Salaev,
• Abstract summary: Affixes play an important role in the morphological analysis of words, by adding additional meanings and grammatical functions to words. This paper present modeling of the morphological analysis of Uzbek words, including stemming, lemmatizing, and the extraction of morphological information. The developed tool based on the proposed model is available as a web-based application and an open-source Python library.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As Uzbek language is agglutinative, has many morphological features which words formed by combining root and affixes. Affixes play an important role in the morphological analysis of words, by adding additional meanings and grammatical functions to words. Inflectional endings are utilized to express various morphological features within the language. This feature introduces numerous possibilities for word endings, thereby significantly expanding the word vocabulary and exacerbating issues related to data sparsity in statistical models. This paper present modeling of the morphological analysis of Uzbek words, including stemming, lemmatizing, and the extraction of morphological information while considering morpho-phonetic exceptions. Main steps of the model involve developing a complete set of word-ending with assigned morphological information, and additional datasets for morphological analysis. The proposed model was evaluated using a curated test set comprising 5.3K words. Through manual verification of stemming, lemmatizing, and morphological feature corrections carried out by linguistic specialists, it obtained a word-level accuracy of over 91%. The developed tool based on the proposed model is available as a web-based application and an open-source Python library.

Related papers

• Exploring Tokenization Strategies and Vocabulary Sizes for Enhanced Arabic Language Models [0.0]
  This paper examines the impact of tokenization strategies and vocabulary sizes on the performance of Arabic language models. Our study uncovers limited impacts of vocabulary size on model performance while keeping the model size unchanged. Paper's recommendations include refining tokenization strategies to address dialect challenges, enhancing model robustness across diverse linguistic contexts, and expanding datasets to encompass the rich dialect based Arabic.
  arXiv  Detail & Related papers  (2024-03-17T07:44:44Z)
• Morphological Inflection with Phonological Features [7.245355976804435]
  This work explores effects on performance obtained through various ways in which morphological models get access to subcharacter phonological features. We elicit phonemic data from standard graphemic data using language-specific grammars for languages with shallow grapheme-to-phoneme mapping.
  arXiv  Detail & Related papers  (2023-06-21T21:34:39Z)
• UniMorph 4.0: Universal Morphology [104.69846084893298]
  This paper presents the expansions and improvements made on several fronts over the last couple of years. Collaborative efforts by numerous linguists have added 67 new languages, including 30 endangered languages. In light of the last UniMorph release, we also augmented the database with morpheme segmentation for 16 languages.
  arXiv  Detail & Related papers  (2022-05-07T09:19:02Z)
• A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space [61.18554842370824]
  In cross-lingual language models, representations for many different languages live in the same space. We compute a task-based measure of cross-lingual alignment in the form of bitext retrieval performance. We examine a range of linguistic, quasi-linguistic, and training-related features as potential predictors of these alignment metrics.
  arXiv  Detail & Related papers  (2021-09-13T21:05:37Z)
• Morphology Matters: A Multilingual Language Modeling Analysis [8.791030561752384]
  Prior studies disagree on whether inflectional morphology makes languages harder to model. We compile a larger corpus of 145 Bible translations in 92 languages and a larger number of typological features. Several morphological measures are significantly associated with higher surprisal when LSTM models are trained with BPE-segmented data.
  arXiv  Detail & Related papers  (2020-12-11T11:55:55Z)
• Morphologically Aware Word-Level Translation [82.59379608647147]
  We propose a novel morphologically aware probability model for bilingual lexicon induction. Our model exploits the basic linguistic intuition that the lexeme is the key lexical unit of meaning.
  arXiv  Detail & Related papers  (2020-11-15T17:54:49Z)
• Morphological Disambiguation from Stemming Data [1.2183405753834562]
  Kinyarwanda, a morphologically rich language, currently lacks tools for automated morphological analysis. We learn to morphologically disambiguate Kinyarwanda verbal forms from a new stemming dataset collected through crowd-sourcing. Our experiments reveal that inflectional properties of stems and morpheme association rules are the most discriminative features for disambiguation.
  arXiv  Detail & Related papers  (2020-11-11T01:44:09Z)
• Grounded Compositional Outputs for Adaptive Language Modeling [59.02706635250856]
  A language model's vocabulary$-$typically selected before training and permanently fixed later$-$affects its size. We propose a fully compositional output embedding layer for language models. To our knowledge, the result is the first word-level language model with a size that does not depend on the training vocabulary.
  arXiv  Detail & Related papers  (2020-09-24T07:21:14Z)
• Neural disambiguation of lemma and part of speech in morphologically rich languages [0.6346772579930928]
  We consider the problem of disambiguating the lemma and part of speech of ambiguous words in morphologically rich languages. We propose a method for disambiguating ambiguous words in context, using a large un-annotated corpus of text, and a morphological analyser.
  arXiv  Detail & Related papers  (2020-07-12T21:48:52Z)
• Evaluating Transformer-Based Multilingual Text Classification [55.53547556060537]
  We argue that NLP tools perform unequally across languages with different syntactic and morphological structures. We calculate word order and morphological similarity indices to aid our empirical study.
  arXiv  Detail & Related papers  (2020-04-29T03:34:53Z)
• A Simple Joint Model for Improved Contextual Neural Lemmatization [60.802451210656805]
  We present a simple joint neural model for lemmatization and morphological tagging that achieves state-of-the-art results on 20 languages. Our paper describes the model in addition to training and decoding procedures.
  arXiv  Detail & Related papers  (2019-04-04T02:03:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.