An Evaluation of Recent Neural Sequence Tagging Models in Turkish Named Entity Recognition

• URL: http://arxiv.org/abs/2005.07692v2
• Date: Mon, 18 May 2020 05:53:16 GMT
• Title: An Evaluation of Recent Neural Sequence Tagging Models in Turkish Named Entity Recognition
• Authors: Gizem Aras, Didem Makaroglu, Seniz Demir, Altan Cakir
• Abstract summary: We propose a transformer-based network with a conditional random field layer that leads to the state-of-the-art result. Our study contributes to the literature that quantifies the impact of transfer learning on processing morphologically rich languages.
• Score: 5.161531917413708
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Named entity recognition (NER) is an extensively studied task that extracts and classifies named entities in a text. NER is crucial not only in downstream language processing applications such as relation extraction and question answering but also in large scale big data operations such as real-time analysis of online digital media content. Recent research efforts on Turkish, a less studied language with morphologically rich nature, have demonstrated the effectiveness of neural architectures on well-formed texts and yielded state-of-the art results by formulating the task as a sequence tagging problem. In this work, we empirically investigate the use of recent neural architectures (Bidirectional long short-term memory and Transformer-based networks) proposed for Turkish NER tagging in the same setting. Our results demonstrate that transformer-based networks which can model long-range context overcome the limitations of BiLSTM networks where different input features at the character, subword, and word levels are utilized. We also propose a transformer-based network with a conditional random field (CRF) layer that leads to the state-of-the-art result (95.95\% f-measure) on a common dataset. Our study contributes to the literature that quantifies the impact of transfer learning on processing morphologically rich languages.

Related papers

• A Sentiment Analysis of Medical Text Based on Deep Learning [1.8130068086063336]
  This paper focuses on the medical domain, using bidirectional encoder representations from transformers (BERT) as the basic pre-trained model. Experiments and analyses were conducted on the METS-CoV dataset to explore the training performance after integrating different deep learning networks. CNN models outperform other networks when trained on smaller medical text datasets in combination with pre-trained models like BERT.
  arXiv  Detail & Related papers  (2024-04-16T12:20:49Z)
• Convolutional Neural Networks for Sentiment Analysis on Weibo Data: A Natural Language Processing Approach [0.228438857884398]
  This study addresses the complex task of sentiment analysis on a dataset of 119,988 original tweets from Weibo using a Convolutional Neural Network (CNN) A CNN-based model was utilized, leveraging word embeddings for feature extraction, and trained to perform sentiment classification. The model achieved a macro-average F1-score of approximately 0.73 on the test set, showing balanced performance across positive, neutral, and negative sentiments.
  arXiv  Detail & Related papers  (2023-07-13T03:02:56Z)
• A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks [60.38369406877899]
  Transformer is a deep neural network that employs a self-attention mechanism to comprehend the contextual relationships within sequential data. transformer models excel in handling long dependencies between input sequence elements and enable parallel processing. Our survey encompasses the identification of the top five application domains for transformer-based models.
  arXiv  Detail & Related papers  (2023-06-11T23:13:51Z)
• Initial Study into Application of Feature Density and Linguistically-backed Embedding to Improve Machine Learning-based Cyberbullying Detection [54.83707803301847]
  The research was conducted on a Formspring dataset provided in a Kaggle competition on automatic cyberbullying detection. The study confirmed the effectiveness of Neural Networks in cyberbullying detection and the correlation between classifier performance and Feature Density.
  arXiv  Detail & Related papers  (2022-06-04T03:17:15Z)
• A Comparative Study of Transformers on Word Sense Disambiguation [0.0]
  This paper presents a comparative study on the contextualization power of neural network-based embedding systems. We evaluate their contextualization power using two sample Word Sense Disambiguation (WSD) tasks, SensEval-2 and SensEval-3. Experimental results show that the proposed techniques also achieve superior results over the current state-of-the-art on both the WSD tasks.
  arXiv  Detail & Related papers  (2021-11-30T14:10:22Z)
• Improving Classifier Training Efficiency for Automatic Cyberbullying Detection with Feature Density [58.64907136562178]
  We study the effectiveness of Feature Density (FD) using different linguistically-backed feature preprocessing methods. We hypothesise that estimating dataset complexity allows for the reduction of the number of required experiments. The difference in linguistic complexity of datasets allows us to additionally discuss the efficacy of linguistically-backed word preprocessing.
  arXiv  Detail & Related papers  (2021-11-02T15:48:28Z)
• Context Decoupling Augmentation for Weakly Supervised Semantic Segmentation [53.49821324597837]
  Weakly supervised semantic segmentation is a challenging problem that has been deeply studied in recent years. We present a Context Decoupling Augmentation ( CDA) method to change the inherent context in which the objects appear. To validate the effectiveness of the proposed method, extensive experiments on PASCAL VOC 2012 dataset with several alternative network architectures demonstrate that CDA can boost various popular WSSS methods to the new state-of-the-art by a large margin.
  arXiv  Detail & Related papers  (2021-03-02T15:05:09Z)
• A Novel Deep Learning Method for Textual Sentiment Analysis [3.0711362702464675]
  This paper proposes a convolutional neural network integrated with a hierarchical attention layer to extract informative words. The proposed model has higher classification accuracy and can extract informative words. Applying incremental transfer learning can significantly enhance the classification performance.
  arXiv  Detail & Related papers  (2021-02-23T12:11:36Z)
• On Long-Tailed Phenomena in Neural Machine Translation [50.65273145888896]
  State-of-the-art Neural Machine Translation (NMT) models struggle with generating low-frequency tokens. We propose a new loss function, the Anti-Focal loss, to better adapt model training to the structural dependencies of conditional text generation. We show the efficacy of the proposed technique on a number of Machine Translation (MT) datasets, demonstrating that it leads to significant gains over cross-entropy.
  arXiv  Detail & Related papers  (2020-10-10T07:00:57Z)
• Probing Linguistic Features of Sentence-Level Representations in Neural Relation Extraction [80.38130122127882]
  We introduce 14 probing tasks targeting linguistic properties relevant to neural relation extraction (RE) We use them to study representations learned by more than 40 different encoder architecture and linguistic feature combinations trained on two datasets. We find that the bias induced by the architecture and the inclusion of linguistic features are clearly expressed in the probing task performance.
  arXiv  Detail & Related papers  (2020-04-17T09:17:40Z)
• TNT-KID: Transformer-based Neural Tagger for Keyword Identification [7.91883337742071]
  We present a novel algorithm for keyword identification called Transformer-based Neural Tagger for Keyword IDentification (TNT-KID) By adapting the transformer architecture for a specific task at hand and leveraging language model pretraining on a domain specific corpus, the model is capable of overcoming deficiencies of both supervised and unsupervised state-of-the-art approaches to keyword extraction.
  arXiv  Detail & Related papers  (2020-03-20T09:55:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.