eFontes. Part of Speech Tagging and Lemmatization of Medieval Latin Texts.A Cross-Genre Survey

• URL: http://arxiv.org/abs/2407.00418v1
• Date: Sat, 29 Jun 2024 11:59:20 GMT
• Title: eFontes. Part of Speech Tagging and Lemmatization of Medieval Latin Texts.A Cross-Genre Survey
• Authors: Krzysztof Nowak, Jędrzej Ziębura, Krzysztof Wróbel, Aleksander Smywiński-Pohl,
• Abstract summary: The eFontes models were trained on Universal Dependencies (UD) corpora and the newly developed eFontes corpus of Polish Medieval Latin. The models achieved high accuracy rates: lemmatization at 92.60%, part-of-speech tagging at 83.29%, and morphological feature determination at 88.57%.
• Score: 41.94295877935867
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study introduces the eFontes models for automatic linguistic annotation of Medieval Latin texts, focusing on lemmatization, part-of-speech tagging, and morphological feature determination. Using the Transformers library, these models were trained on Universal Dependencies (UD) corpora and the newly developed eFontes corpus of Polish Medieval Latin. The research evaluates the models' performance, addressing challenges such as orthographic variations and the integration of Latinized vernacular terms. The models achieved high accuracy rates: lemmatization at 92.60%, part-of-speech tagging at 83.29%, and morphological feature determination at 88.57%. The findings underscore the importance of high-quality annotated corpora and propose future enhancements, including extending the models to Named Entity Recognition.

Related papers

• UzMorphAnalyser: A Morphological Analysis Model for the Uzbek Language Using Inflectional Endings [0.0]
  Affixes play an important role in the morphological analysis of words, by adding additional meanings and grammatical functions to words. This paper present modeling of the morphological analysis of Uzbek words, including stemming, lemmatizing, and the extraction of morphological information. The developed tool based on the proposed model is available as a web-based application and an open-source Python library.
  arXiv  Detail & Related papers  (2024-05-23T05:06:55Z)
• LiMe: a Latin Corpus of Late Medieval Criminal Sentences [39.26357402982764]
  We present the LiMe dataset, a corpus of 325 documents extracted from a series of medieval manuscripts called Libri sententiarum potestatis Mediolani.
  arXiv  Detail & Related papers  (2024-04-19T12:06:28Z)
• Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Model [58.27176041092891]
  Recent research indicates that pretraining cross-lingual language models on large-scale unlabeled texts yields significant performance improvements. We propose a novel unsupervised feature decomposition method that can automatically extract domain-specific features from the entangled pretrained cross-lingual representations. Our proposed model leverages mutual information estimation to decompose the representations computed by a cross-lingual model into domain-invariant and domain-specific parts.
  arXiv  Detail & Related papers  (2020-11-23T16:00:42Z)
• Morphologically Aware Word-Level Translation [82.59379608647147]
  We propose a novel morphologically aware probability model for bilingual lexicon induction. Our model exploits the basic linguistic intuition that the lexeme is the key lexical unit of meaning.
  arXiv  Detail & Related papers  (2020-11-15T17:54:49Z)
• Grounded Compositional Outputs for Adaptive Language Modeling [59.02706635250856]
  A language model's vocabulary$-$typically selected before training and permanently fixed later$-$affects its size. We propose a fully compositional output embedding layer for language models. To our knowledge, the result is the first word-level language model with a size that does not depend on the training vocabulary.
  arXiv  Detail & Related papers  (2020-09-24T07:21:14Z)
• Latin BERT: A Contextual Language Model for Classical Philology [7.513100214864645]
  We present Latin BERT, a contextual language model for the Latin language. It was trained on 642.7 million words from a variety of sources spanning the Classical era to the 21st century.
  arXiv  Detail & Related papers  (2020-09-21T17:47:44Z)
• Labeling Explicit Discourse Relations using Pre-trained Language Models [0.0]
  State-of-the-art models achieve slightly above 45% of F-score by using hand-crafted features. We find that the pre-trained language models, when finetuned, are powerful enough to replace the linguistic features. This is the first time when a model outperforms the knowledge intensive models without employing any linguistic features.
  arXiv  Detail & Related papers  (2020-06-21T17:18:01Z)
• The Frankfurt Latin Lexicon: From Morphological Expansion and Word Embeddings to SemioGraphs [97.8648124629697]
  The article argues for a more comprehensive understanding of lemmatization, encompassing classical machine learning as well as intellectual post-corrections and, in particular, human interpretation processes based on graph representations of the underlying lexical resources.
  arXiv  Detail & Related papers  (2020-05-21T17:16:53Z)
• Limits of Detecting Text Generated by Large-Scale Language Models [65.46403462928319]
  Some consider large-scale language models that can generate long and coherent pieces of text as dangerous, since they may be used in misinformation campaigns. Here we formulate large-scale language model output detection as a hypothesis testing problem to classify text as genuine or generated.
  arXiv  Detail & Related papers  (2020-02-09T19:53:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.