CKMorph: A Comprehensive Morphological Analyzer for Central Kurdish

• URL: http://arxiv.org/abs/2109.08615v1
• Date: Fri, 17 Sep 2021 15:45:27 GMT
• Title: CKMorph: A Comprehensive Morphological Analyzer for Central Kurdish
• Abstract summary: A morphological analyzer is a significant component of many natural language processing applications especially for morphologically rich languages. This paper introduces a comprehensive morphological analyzer for Central Kurdish (CK), a low-resourced language with a rich morphology. CKMorph was able to correctly analyze 95.9% of the accuracy test set, containing 1,000 CK words morphologically analyzed according to the context.
• Score: 4.226093500082746
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A morphological analyzer, which is a significant component of many natural language processing applications especially for morphologically rich languages, divides an input word into all its composing morphemes and identifies their morphological roles. In this paper, we introduce a comprehensive morphological analyzer for Central Kurdish (CK), a low-resourced language with a rich morphology. Building upon the limited existing literature, we first assembled and systematically categorized a comprehensive collection of the morphological and morphophonological rules of the language. Additionally, we collected and manually labeled a generative lexicon containing nearly 10,000 verb, noun and adjective stems, named entities, and other types of word stems. We used these rule sets and resources to implement CKMorph Analyzer based on finite-state transducers. In order to provide a benchmark for future research, we collected, manually labeled, and publicly shared test sets for evaluating accuracy and coverage of the analyzer. CKMorph was able to correctly analyze 95.9% of the accuracy test set, containing 1,000 CK words morphologically analyzed according to the context. Moreover, CKMorph gave at least one analysis for 95.5% of 4.22M CK tokens of the coverage test set. The demonstration of the application and resources including CK verb database and test sets are openly accessible at https://github.com/CKMorph.

Related papers

• MorfFlex: Handling Rich Morphology [1.4766181043669164]
  We present MorfFlex, a morphological dictionary architecture suitable for languages with extensive regularity in both inflection and derivation.<n>As the primary example of MorfFlex in use we introduce MorfFlex CZ, a morphological dictionary of Czech.<n>It is distributed as a simple, unstructured list of wordform, lemma, tag> triplets, however, its manually maintained, unpublished source files and conversion scripts encode a sophisticated system of inflectional and derivational patterns.
  arXiv  Detail & Related papers  (2026-06-23T09:56:58Z)
• Optimal Turkish Subword Strategies at Scale: Systematic Evaluation of Data, Vocabulary, Morphology Interplay [4.061135251278187]
  Tokenization is a pivotal design choice for neural language modeling in morphologically rich languages.<n>We present the first comprehensive, principled study of Turkish subword tokenization.
  arXiv  Detail & Related papers  (2026-02-06T18:41:14Z)
• RusLICA: A Russian-Language Platform for Automated Linguistic Inquiry and Category Analysis [0.0]
  The paper describes the process of mapping lemmas to 42 psycholinguistic categories and the implementation of the analyzer as part of RusLICA web service.
  arXiv  Detail & Related papers  (2026-01-28T05:43:40Z)
• Practical Machine Learning for Aphasic Discourse Analysis [0.0]
  This study evaluated five machine learning models for reliable identification of Correct Information Units (CIUs) during a picture description task.<n>The baseline model training produced a high accuracy across transcripts for word vs non-word, with all models achieving near perfect performance.<n>In contrast, CIU vs non-CIU showed a greater variability, with the k-nearest neighbor (k-NN) model the highest accuracy (0.824) and second highest AUC (0.787)
  arXiv  Detail & Related papers  (2025-11-12T11:42:17Z)
• MoVoC: Morphology-Aware Subword Construction for Geez Script Languages [7.7761618950496265]
  Subword-based tokenization methods often fail to preserve morphological boundaries.<n>We present MoVoC (Morpheme-aware Subword Vocabulary Construction) and train MoVoC-Tok, a tokenizer that integrates supervised morphological analysis into the subword vocabulary.
  arXiv  Detail & Related papers  (2025-09-10T17:45:10Z)
• Urdu Dependency Parsing and Treebank Development: A Syntactic and Morphological Perspective [0.0]
  We use dependency parsing to analyze news articles in Urdu. We achieve a best-labeled accuracy (LA) of 70% and an unlabeled attachment score (UAS) of 84%.
  arXiv  Detail & Related papers  (2024-06-13T19:30:32Z)
• UzMorphAnalyser: A Morphological Analysis Model for the Uzbek Language Using Inflectional Endings [0.0]
  Affixes play an important role in the morphological analysis of words, by adding additional meanings and grammatical functions to words. This paper present modeling of the morphological analysis of Uzbek words, including stemming, lemmatizing, and the extraction of morphological information. The developed tool based on the proposed model is available as a web-based application and an open-source Python library.
  arXiv  Detail & Related papers  (2024-05-23T05:06:55Z)
• Homonym Sense Disambiguation in the Georgian Language [49.1574468325115]
  This research proposes a novel approach to the Word Sense Disambiguation (WSD) task in the Georgian language. It is based on supervised fine-tuning of a pre-trained Large Language Model (LLM) on a dataset formed by filtering the Georgian Common Crawls corpus.
  arXiv  Detail & Related papers  (2024-04-24T21:48:43Z)
• UniMorph 4.0: Universal Morphology [104.69846084893298]
  This paper presents the expansions and improvements made on several fronts over the last couple of years. Collaborative efforts by numerous linguists have added 67 new languages, including 30 endangered languages. In light of the last UniMorph release, we also augmented the database with morpheme segmentation for 16 languages.
  arXiv  Detail & Related papers  (2022-05-07T09:19:02Z)
• Few-Shot Cross-lingual Transfer for Coarse-grained De-identification of Code-Mixed Clinical Texts [56.72488923420374]
  Pre-trained language models (LMs) have shown great potential for cross-lingual transfer in low-resource settings. We show the few-shot cross-lingual transfer property of LMs for named recognition (NER) and apply it to solve a low-resource and real-world challenge of code-mixed (Spanish-Catalan) clinical notes de-identification in the stroke.
  arXiv  Detail & Related papers  (2022-04-10T21:46:52Z)
• An analysis of full-size Russian complexly NER labelled corpus of Internet user reviews on the drugs based on deep learning and language neural nets [94.37521840642141]
  We present the full-size Russian complexly NER-labeled corpus of Internet user reviews. A set of advanced deep learning neural networks is used to extract pharmacologically meaningful entities from Russian texts.
  arXiv  Detail & Related papers  (2021-04-30T19:46:24Z)
• Evaluating the Morphosyntactic Well-formedness of Generated Texts [88.20502652494521]
  We propose L'AMBRE -- a metric to evaluate the morphosyntactic well-formedness of text. We show the effectiveness of our metric on the task of machine translation through a diachronic study of systems translating into morphologically-rich languages.
  arXiv  Detail & Related papers  (2021-03-30T18:02:58Z)
• Evaluation of Morphological Embeddings for the Russian Language [0.0]
  morphology-based embeddings trained with Skipgram objective do not outperform existing embedding model -- FastText. A more complex, but morphology unaware model, BERT, allows to achieve significantly greater performance on the tasks that presumably require understanding of a word's morphology.
  arXiv  Detail & Related papers  (2021-03-11T11:59:11Z)
• Morphology Matters: A Multilingual Language Modeling Analysis [8.791030561752384]
  Prior studies disagree on whether inflectional morphology makes languages harder to model. We compile a larger corpus of 145 Bible translations in 92 languages and a larger number of typological features. Several morphological measures are significantly associated with higher surprisal when LSTM models are trained with BPE-segmented data.
  arXiv  Detail & Related papers  (2020-12-11T11:55:55Z)
• Morphological Disambiguation from Stemming Data [1.2183405753834562]
  Kinyarwanda, a morphologically rich language, currently lacks tools for automated morphological analysis. We learn to morphologically disambiguate Kinyarwanda verbal forms from a new stemming dataset collected through crowd-sourcing. Our experiments reveal that inflectional properties of stems and morpheme association rules are the most discriminative features for disambiguation.
  arXiv  Detail & Related papers  (2020-11-11T01:44:09Z)
• Evaluating Transformer-Based Multilingual Text Classification [55.53547556060537]
  We argue that NLP tools perform unequally across languages with different syntactic and morphological structures. We calculate word order and morphological similarity indices to aid our empirical study.
  arXiv  Detail & Related papers  (2020-04-29T03:34:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.