Dependency Annotation of Ottoman Turkish with Multilingual BERT

• URL: http://arxiv.org/abs/2402.14743v2
• Date: Thu, 22 Aug 2024 11:29:42 GMT
• Title: Dependency Annotation of Ottoman Turkish with Multilingual BERT
• Authors: Şaziye Betül Özateş, Tarık Emre Tıraş, Efe Eren Genç, Esma Fatıma Bilgin Taşdemir,
• Abstract summary: This study introduces a pretrained large language model-based annotation methodology for the first dency treebank in Ottoman Turkish. The resulting treebank will facilitate automated analysis of Ottoman Turkish documents, unlocking the linguistic richness embedded in this historical heritage.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This study introduces a pretrained large language model-based annotation methodology for the first de dency treebank in Ottoman Turkish. Our experimental results show that, iteratively, i) pseudo-annotating data using a multilingual BERT-based parsing model, ii) manually correcting the pseudo-annotations, and iii) fine-tuning the parsing model with the corrected annotations, we speed up and simplify the challenging dependency annotation process. The resulting treebank, that will be a part of the Universal Dependencies (UD) project, will facilitate automated analysis of Ottoman Turkish documents, unlocking the linguistic richness embedded in this historical heritage.

Related papers

• Localizing Factual Inconsistencies in Attributable Text Generation [91.981439746404]
  We introduce QASemConsistency, a new formalism for localizing factual inconsistencies in attributable text generation. We first demonstrate the effectiveness of the QASemConsistency methodology for human annotation. We then implement several methods for automatically detecting localized factual inconsistencies.
  arXiv  Detail & Related papers  (2024-10-09T22:53:48Z)
• Adapting PromptORE for Modern History: Information Extraction from Hispanic Monarchy Documents of the XVIth Century [2.490441444378203]
  We introduce an adaptation of PromptORE to extract relations from specialized documents, namely digital transcripts of trials from the Spanish Inquisition. Our approach involves fine-tuning transformer models with their pretraining objective on the data they will perform inference. Our results show a substantial improvement in accuracy -up to a 50% improvement with our Biased PromptORE models.
  arXiv  Detail & Related papers  (2024-05-24T13:39:47Z)
• Rethinking Masked Language Modeling for Chinese Spelling Correction [70.85829000570203]
  We study Chinese Spelling Correction (CSC) as a joint decision made by two separate models: a language model and an error model. We find that fine-tuning BERT tends to over-fit the error model while under-fit the language model, resulting in poor generalization to out-of-distribution error patterns. We demonstrate that a very simple strategy, randomly masking 20% non-error tokens from the input sequence during fine-tuning is sufficient for learning a much better language model without sacrificing the error model.
  arXiv  Detail & Related papers  (2023-05-28T13:19:12Z)
• Beyond Contrastive Learning: A Variational Generative Model for Multilingual Retrieval [109.62363167257664]
  We propose a generative model for learning multilingual text embeddings. Our model operates on parallel data in $N$ languages. We evaluate this method on a suite of tasks including semantic similarity, bitext mining, and cross-lingual question retrieval.
  arXiv  Detail & Related papers  (2022-12-21T02:41:40Z)
• A Unified Neural Network Model for Readability Assessment with Feature Projection and Length-Balanced Loss [17.213602354715956]
  We propose a BERT-based model with feature projection and length-balanced loss for readability assessment. Our model achieves state-of-the-art performances on two English benchmark datasets and one dataset of Chinese textbooks.
  arXiv  Detail & Related papers  (2022-10-19T05:33:27Z)
• CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation [113.99145386490639]
  Cross-lingual NER can transfer knowledge between languages via aligned cross-lingual representations or machine translation results. We propose a Cross-lingual Entity Projection framework (CROP) to enable zero-shot cross-lingual NER. We adopt a multilingual labeled sequence translation model to project the tagged sequence back to the target language and label the target raw sentence.
  arXiv  Detail & Related papers  (2022-10-13T13:32:36Z)
• Unsupervised Paraphrasing with Pretrained Language Models [85.03373221588707]
  We propose a training pipeline that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting. Our recipe consists of task-adaptation, self-supervision, and a novel decoding algorithm named Dynamic Blocking. We show with automatic and human evaluations that our approach achieves state-of-the-art performance on both the Quora Question Pair and the ParaNMT datasets.
  arXiv  Detail & Related papers  (2020-10-24T11:55:28Z)
• Entity and Evidence Guided Relation Extraction for DocRED [33.69481141963074]
  We pro-pose a joint training frameworkE2GRE(Entity and Evidence Guided Relation Extraction)for this task. We introduce entity-guided sequences as inputs to a pre-trained language model (e.g. BERT, RoBERTa) These entity-guided sequences help a pre-trained language model (LM) to focus on areas of the document related to the entity. We evaluate our E2GRE approach on DocRED, a recently released large-scale dataset for relation extraction.
  arXiv  Detail & Related papers  (2020-08-27T17:41:23Z)
• Analysis of the Penn Korean Universal Dependency Treebank (PKT-UD): Manual Revision to Build Robust Parsing Model in Korean [15.899449418195106]
  We first open on important issues regarding the Penn Korean Universal Treebank (PKT-UD) We address these issues by revising the entire corpus manually with the aim of producing cleaner UD annotations. For compatibility to the rest of UD corpora, we extensively revise the part-of-speech tags and the dependency relations.
  arXiv  Detail & Related papers  (2020-05-26T17:46:46Z)
• Coreferential Reasoning Learning for Language Representation [88.14248323659267]
  We present CorefBERT, a novel language representation model that can capture the coreferential relations in context. The experimental results show that, compared with existing baseline models, CorefBERT can achieve significant improvements consistently on various downstream NLP tasks.
  arXiv  Detail & Related papers  (2020-04-15T03:57:45Z)
• Resources for Turkish Dependency Parsing: Introducing the BOUN Treebank and the BoAT Annotation Tool [0.0]
  We introduce the resources that we developed for Turkish dependency parsing, which include a novel manually annotated treebank (BOUN Treebank) Decisions regarding the annotation of the BOUN Treebank were made in line with the Universal Dependencies (UD) framework. We report the results of a state-of-the-art dependency annotation obtained over the BOUN Treebank as well as two other treebanks in Turkish.
  arXiv  Detail & Related papers  (2020-02-24T17:59:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.