New Directions in Text Classification Research: Maximizing The Performance of Sentiment Classification from Limited Data

• URL: http://arxiv.org/abs/2407.05627v1
• Date: Mon, 8 Jul 2024 05:42:29 GMT
• Title: New Directions in Text Classification Research: Maximizing The Performance of Sentiment Classification from Limited Data
• Authors: Surya Agustian, Muhammad Irfan Syah, Nurul Fatiara, Rahmad Abdillah,
• Abstract summary: A benchmark dataset is provided for training and testing data on the issue of Kaesang Pangarep's appointment as Chairman of PSI. The official score used is the F1-score, which balances precision and recall among the three classes, positive, negative, and neutral. Both scoring (baseline and optimized) use the SVM method, which is widely reported as the state-of-the-art in conventional machine learning methods.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The stakeholders' needs in sentiment analysis for various issues, whether positive or negative, are speed and accuracy. One new challenge in sentiment analysis tasks is the limited training data, which often leads to suboptimal machine learning models and poor performance on test data. This paper discusses the problem of text classification based on limited training data (300 to 600 samples) into three classes: positive, negative, and neutral. A benchmark dataset is provided for training and testing data on the issue of Kaesang Pangarep's appointment as Chairman of PSI. External data for aggregation and augmentation purposes are provided, consisting of two datasets: the topic of Covid Vaccination sentiment and an open topic. The official score used is the F1-score, which balances precision and recall among the three classes, positive, negative, and neutral. A baseline score is provided as a reference for researchers for unoptimized classification methods. The optimized score is provided as a reference for the target score to be achieved by any proposed method. Both scoring (baseline and optimized) use the SVM method, which is widely reported as the state-of-the-art in conventional machine learning methods. The F1-scores achieved by the baseline and optimized methods are 40.83% and 51.28%, respectively.

Related papers

• Self-Training with Pseudo-Label Scorer for Aspect Sentiment Quad Prediction [54.23208041792073]
  Aspect Sentiment Quad Prediction (ASQP) aims to predict all quads (aspect term, aspect category, opinion term, sentiment polarity) for a given review. A key challenge in the ASQP task is the scarcity of labeled data, which limits the performance of existing methods. We propose a self-training framework with a pseudo-label scorer, wherein a scorer assesses the match between reviews and their pseudo-labels.
  arXiv  Detail & Related papers  (2024-06-26T05:30:21Z)
• Class-Imbalanced Semi-Supervised Learning for Large-Scale Point Cloud Semantic Segmentation via Decoupling Optimization [64.36097398869774]
  Semi-supervised learning (SSL) has been an active research topic for large-scale 3D scene understanding. The existing SSL-based methods suffer from severe training bias due to class imbalance and long-tail distributions of the point cloud data. We introduce a new decoupling optimization framework, which disentangles feature representation learning and classifier in an alternative optimization manner to shift the bias decision boundary effectively.
  arXiv  Detail & Related papers  (2024-01-13T04:16:40Z)
• Blueprinting the Future: Automatic Item Categorization using Hierarchical Zero-Shot and Few-Shot Classifiers [6.907552533477328]
  This study unveils a novel approach employing the zero-shot and few-shot Generative Pretrained Transformer (GPT) for hierarchical item categorization. The hierarchical nature of examination blueprints is navigated seamlessly, allowing for a tiered classification of items across multiple levels. An initial simulation with artificial data demonstrates the efficacy of this method, achieving an average accuracy of 92.91% measured by the F1 score.
  arXiv  Detail & Related papers  (2023-12-06T15:51:49Z)
• DST-Det: Simple Dynamic Self-Training for Open-Vocabulary Object Detection [72.25697820290502]
  This work introduces a straightforward and efficient strategy to identify potential novel classes through zero-shot classification. We refer to this approach as the self-training strategy, which enhances recall and accuracy for novel classes without requiring extra annotations, datasets, and re-training. Empirical evaluations on three datasets, including LVIS, V3Det, and COCO, demonstrate significant improvements over the baseline performance.
  arXiv  Detail & Related papers  (2023-10-02T17:52:24Z)
• Revisiting Long-tailed Image Classification: Survey and Benchmarks with New Evaluation Metrics [88.39382177059747]
  A corpus of metrics is designed for measuring the accuracy, robustness, and bounds of algorithms for learning with long-tailed distribution. Based on our benchmarks, we re-evaluate the performance of existing methods on CIFAR10 and CIFAR100 datasets.
  arXiv  Detail & Related papers  (2023-02-03T02:40:54Z)
• CAFA: Class-Aware Feature Alignment for Test-Time Adaptation [50.26963784271912]
  Test-time adaptation (TTA) aims to address this challenge by adapting a model to unlabeled data at test time. We propose a simple yet effective feature alignment loss, termed as Class-Aware Feature Alignment (CAFA), which simultaneously encourages a model to learn target representations in a class-discriminative manner.
  arXiv  Detail & Related papers  (2022-06-01T03:02:07Z)
• CheckSel: Efficient and Accurate Data-valuation Through Online Checkpoint Selection [3.321404824316694]
  We propose a novel 2-phase solution to the problem of data valuation and subset selection. Phase 1 selects representative checkpoints from an SGD-like training algorithm, which are used in phase-2 to estimate the approximate training data values. Experimental results show the proposed algorithm outperforms recent baseline methods by up to 30% in terms of test accuracy.
  arXiv  Detail & Related papers  (2022-03-14T02:06:52Z)
• Semi-Supervised Semantic Segmentation via Adaptive Equalization Learning [20.66927648806676]
  We propose a novel framework for semi-supervised semantic segmentation, named adaptive equalization learning (AEL) AEL balances the training of well and badly performed categories, with a confidence bank to track category-wise performance. AEL outperforms the state-of-the-art methods by a large margin on the Cityscapes and Pascal VOC benchmarks.
  arXiv  Detail & Related papers  (2021-10-11T17:59:55Z)
• A new weakly supervised approach for ALS point cloud semantic segmentation [1.4620086904601473]
  We propose a deep-learning based weakly supervised framework for semantic segmentation of ALS point clouds. We exploit potential information from unlabeled data subject to incomplete and sparse labels. Our method achieves an overall accuracy of 83.0% and an average F1 score of 70.0%, which have increased by 6.9% and 12.8% respectively.
  arXiv  Detail & Related papers  (2021-10-04T14:00:23Z)
• Out-of-Vocabulary Entities in Link Prediction [1.9036571490366496]
  Link prediction is often used as a proxy to evaluate the quality of embeddings. As benchmarks are crucial for the fair comparison of algorithms, ensuring their quality is tantamount to providing a solid ground for developing better solutions. We provide an implementation of an approach for spotting and removing such entities and provide corrected versions of the datasets WN18RR, FB15K-237, and YAGO3-10.
  arXiv  Detail & Related papers  (2021-05-26T12:58:18Z)
• Uncertainty-aware Self-training for Text Classification with Few Labels [54.13279574908808]
  We study self-training as one of the earliest semi-supervised learning approaches to reduce the annotation bottleneck. We propose an approach to improve self-training by incorporating uncertainty estimates of the underlying neural network. We show our methods leveraging only 20-30 labeled samples per class for each task for training and for validation can perform within 3% of fully supervised pre-trained language models.
  arXiv  Detail & Related papers  (2020-06-27T08:13:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.