Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers on Benchmarks and Real-world Datasets

• URL: http://arxiv.org/abs/2211.16878v3
• Date: Fri, 11 Aug 2023 11:25:49 GMT
• Title: Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers on Benchmarks and Real-world Datasets
• Authors: Fabian Karl and Ansgar Scherp
• Abstract summary: Short text classification is a crucial and challenging aspect of Natural Language Processing. In recent short text research, State of the Art (SOTA) methods for traditional text classification have been unexploited. Our experiments unambiguously demonstrate that Transformers achieve SOTA accuracy on short text classification tasks.
• Score: 2.9443230571766854
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Short text classification is a crucial and challenging aspect of Natural Language Processing. For this reason, there are numerous highly specialized short text classifiers. However, in recent short text research, State of the Art (SOTA) methods for traditional text classification, particularly the pure use of Transformers, have been unexploited. In this work, we examine the performance of a variety of short text classifiers as well as the top performing traditional text classifier. We further investigate the effects on two new real-world short text datasets in an effort to address the issue of becoming overly dependent on benchmark datasets with a limited number of characteristics. Our experiments unambiguously demonstrate that Transformers achieve SOTA accuracy on short text classification tasks, raising the question of whether specialized short text techniques are necessary.

Related papers

• Exploring the Limitations of Detecting Machine-Generated Text [29.06307663406079]
  We critically examine the classification performance for detecting machine-generated text by evaluating on texts with varying writing styles. We find that classifiers are highly sensitive to stylistic changes and differences in text complexity. We further find that detection systems are particularly susceptible to misclassify easy-to-read texts while they have high performance for complex texts.
  arXiv  Detail & Related papers  (2024-06-16T21:02:02Z)
• Enhancing Scene Text Detectors with Realistic Text Image Synthesis Using Diffusion Models [63.99110667987318]
  We present DiffText, a pipeline that seamlessly blends foreground text with the background's intrinsic features. With fewer text instances, our produced text images consistently surpass other synthetic data in aiding text detectors.
  arXiv  Detail & Related papers  (2023-11-28T06:51:28Z)
• TextFormer: A Query-based End-to-End Text Spotter with Mixed Supervision [61.186488081379]
  We propose TextFormer, a query-based end-to-end text spotter with Transformer architecture. TextFormer builds upon an image encoder and a text decoder to learn a joint semantic understanding for multi-task modeling. It allows for mutual training and optimization of classification, segmentation, and recognition branches, resulting in deeper feature sharing.
  arXiv  Detail & Related papers  (2023-06-06T03:37:41Z)
• Selective Text Augmentation with Word Roles for Low-Resource Text Classification [3.4806267677524896]
  Different words may play different roles in text classification, which inspires us to strategically select the proper roles for text augmentation. In this work, we first identify the relationships between the words in a text and the text category from the perspectives of statistical correlation and semantic similarity. We present a new augmentation technique called STA (Selective Text Augmentation) where different text-editing operations are selectively applied to words with specific roles.
  arXiv  Detail & Related papers  (2022-09-04T08:13:11Z)
• Contextual Text Block Detection towards Scene Text Understanding [85.40898487745272]
  This paper presents contextual text detection, a new setup that detects contextual text blocks (CTBs) for better understanding of texts in scenes. We formulate the new setup by a dual detection task which first detects integral text units and then groups them into a CTB. To this end, we design a novel scene text clustering technique that treats integral text units as tokens and groups them (belonging to the same CTB) into an ordered token sequence.
  arXiv  Detail & Related papers  (2022-07-26T14:59:25Z)
• CORE-Text: Improving Scene Text Detection with Contrastive Relational Reasoning [65.57338873921168]
  Localizing text instances in natural scenes is regarded as a fundamental challenge in computer vision. In this work, we quantitatively analyze the sub-text problem and present a simple yet effective design, COntrastive RElation (CORE) module. We integrate the CORE module into a two-stage text detector of Mask R-CNN and devise our text detector CORE-Text.
  arXiv  Detail & Related papers  (2021-12-14T16:22:25Z)
• What Have Been Learned & What Should Be Learned? An Empirical Study of How to Selectively Augment Text for Classification [0.0]
  We propose STA (Selective Text Augmentation) to selectively augment the text, where the informative, class-indicating words are emphasized but the irrelevant or noisy words are diminished. Experiments on four English and Chinese text classification benchmark datasets demonstrate that STA can substantially outperform the non-selective text augmentation methods.
  arXiv  Detail & Related papers  (2021-09-01T04:03:11Z)
• A Case Study of Spanish Text Transformations for Twitter Sentiment Analysis [1.9694608733361543]
  Sentiment analysis is a text mining task that determines the polarity of a given text, i.e., its positiveness or negativeness. New forms of textual expressions present new challenges to analyze text given the use of slang, orthographic and grammatical errors.
  arXiv  Detail & Related papers  (2021-06-03T17:24:31Z)
• Pretrained Transformers for Text Ranking: BERT and Beyond [53.83210899683987]
  This survey provides an overview of text ranking with neural network architectures known as transformers. The combination of transformers and self-supervised pretraining has been responsible for a paradigm shift in natural language processing.
  arXiv  Detail & Related papers  (2020-10-13T15:20:32Z)
• Text Perceptron: Towards End-to-End Arbitrary-Shaped Text Spotting [49.768327669098674]
  We propose an end-to-end trainable text spotting approach named Text Perceptron. It first employs an efficient segmentation-based text detector that learns the latent text reading order and boundary information. Then a novel Shape Transform Module (abbr. STM) is designed to transform the detected feature regions into regular morphologies.
  arXiv  Detail & Related papers  (2020-02-17T08:07:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.