Bangla Text Classification using Transformers

• URL: http://arxiv.org/abs/2011.04446v1
• Date: Mon, 9 Nov 2020 14:12:07 GMT
• Title: Bangla Text Classification using Transformers
• Authors: Tanvirul Alam, Akib Khan and Firoj Alam
• Abstract summary: Text classification has been one of the earliest problems in NLP. In this work, we fine-tune multilingual Transformer models for Bangla text classification tasks. We obtain the state of the art results on six benchmark datasets, improving upon the previous results by 5-29% accuracy across different tasks.
• Score: 2.3475904942266697
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text classification has been one of the earliest problems in NLP. Over time the scope of application areas has broadened and the difficulty of dealing with new areas (e.g., noisy social media content) has increased. The problem-solving strategy switched from classical machine learning to deep learning algorithms. One of the recent deep neural network architecture is the Transformer. Models designed with this type of network and its variants recently showed their success in many downstream natural language processing tasks, especially for resource-rich languages, e.g., English. However, these models have not been explored fully for Bangla text classification tasks. In this work, we fine-tune multilingual transformer models for Bangla text classification tasks in different domains, including sentiment analysis, emotion detection, news categorization, and authorship attribution. We obtain the state of the art results on six benchmark datasets, improving upon the previous results by 5-29% accuracy across different tasks.

Related papers

• Bangla Grammatical Error Detection Leveraging Transformer-based Token Classification [0.0]
  We study the development of an automated grammar checker in Bangla, the seventh most spoken language in the world. Our approach involves breaking down the task as a token classification problem and utilizing state-of-the-art transformer-based models. Our system is evaluated on a dataset consisting of over 25,000 texts from various sources.
  arXiv  Detail & Related papers  (2024-11-13T05:22:45Z)
• Language Models for Text Classification: Is In-Context Learning Enough? [54.869097980761595]
  Recent foundational language models have shown state-of-the-art performance in many NLP tasks in zero- and few-shot settings. An advantage of these models over more standard approaches is the ability to understand instructions written in natural language (prompts) This makes them suitable for addressing text classification problems for domains with limited amounts of annotated instances.
  arXiv  Detail & Related papers  (2024-03-26T12:47:39Z)
• T3L: Translate-and-Test Transfer Learning for Cross-Lingual Text Classification [50.675552118811]
  Cross-lingual text classification is typically built on large-scale, multilingual language models (LMs) pretrained on a variety of languages of interest. We propose revisiting the classic "translate-and-test" pipeline to neatly separate the translation and classification stages.
  arXiv  Detail & Related papers  (2023-06-08T07:33:22Z)
• To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLP [0.0]
  We investigate three categories of text augmentation methodologies which perform changes on the syntax. We compare them on part-of-speech tagging, dependency parsing and semantic role labeling for a diverse set of language families. Our results suggest that the augmentation techniques can further improve over strong baselines based on mBERT.
  arXiv  Detail & Related papers  (2021-11-18T10:52:48Z)
• Sentiment analysis in tweets: an assessment study from classical to modern text representation models [59.107260266206445]
  Short texts published on Twitter have earned significant attention as a rich source of information. Their inherent characteristics, such as the informal, and noisy linguistic style, remain challenging to many natural language processing (NLP) tasks. This study fulfils an assessment of existing language models in distinguishing the sentiment expressed in tweets by using a rich collection of 22 datasets.
  arXiv  Detail & Related papers  (2021-05-29T21:05:28Z)
• Sentiment Classification in Bangla Textual Content: A Comparative Study [4.2394281761764]
  In this study, we explore several publicly available sentiment labeled datasets and designed classifiers using both classical and deep learning algorithms. Our finding suggests transformer-based models, which have not been explored earlier for Bangla, outperform all other models.
  arXiv  Detail & Related papers  (2020-11-19T21:06:28Z)
• Be More with Less: Hypergraph Attention Networks for Inductive Text Classification [56.98218530073927]
  Graph neural networks (GNNs) have received increasing attention in the research community and demonstrated their promising results on this canonical task. Despite the success, their performance could be largely jeopardized in practice since they are unable to capture high-order interaction between words. We propose a principled model -- hypergraph attention networks (HyperGAT) which can obtain more expressive power with less computational consumption for text representation learning.
  arXiv  Detail & Related papers  (2020-11-01T00:21:59Z)
• Pretrained Transformers for Text Ranking: BERT and Beyond [53.83210899683987]
  This survey provides an overview of text ranking with neural network architectures known as transformers. The combination of transformers and self-supervised pretraining has been responsible for a paradigm shift in natural language processing.
  arXiv  Detail & Related papers  (2020-10-13T15:20:32Z)
• An Evaluation of Recent Neural Sequence Tagging Models in Turkish Named Entity Recognition [5.161531917413708]
  We propose a transformer-based network with a conditional random field layer that leads to the state-of-the-art result. Our study contributes to the literature that quantifies the impact of transfer learning on processing morphologically rich languages.
  arXiv  Detail & Related papers  (2020-05-14T06:54:07Z)
• Deep Learning for Hindi Text Classification: A Comparison [6.8629257716723]
  The research in the classification of morphologically rich and low resource Hindi language written in Devanagari script has been limited due to the absence of large labeled corpus. In this work, we used translated versions of English data-sets to evaluate models based on CNN, LSTM and Attention. The paper also serves as a tutorial for popular text classification techniques.
  arXiv  Detail & Related papers  (2020-01-19T09:29:12Z)
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer [64.22926988297685]
  Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP) In this paper, we explore the landscape of introducing transfer learning techniques for NLP by a unified framework that converts all text-based language problems into a text-to-text format.
  arXiv  Detail & Related papers  (2019-10-23T17:37:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.