Benchmarking PyCaret AutoML Against BiLSTM for Fine-Grained Emotion Classification: A Comparative Study on 20-Class Emotion Detection

• URL: http://arxiv.org/abs/2604.26310v1
• Date: Wed, 29 Apr 2026 05:31:45 GMT
• Title: Benchmarking PyCaret AutoML Against BiLSTM for Fine-Grained Emotion Classification: A Comparative Study on 20-Class Emotion Detection
• Abstract summary: This study benchmarks classical machine learning and deep learning approaches for 20-class emotion classification.<n>The results show that BiLSTM achieves the best overall performance with 89% accuracy and a weighted F1-score of 0.89.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fine-grained emotion classification, which identifies specific emotional states such as happiness, anger, sadness, and fear, remains a challenging task in natural language processing. This study benchmarks classical machine learning and deep learning approaches for 20-class emotion classification using the 20-Emotion Text Classification Dataset containing 79,595 English sentences. On the machine learning side, Logistic Regression, Multinomial Naive Bayes, and Support Vector Machine are evaluated using TF-IDF features. On the deep learning side, Bidirectional Long Short-Term Memory, Gated Recurrent Unit, and a lightweight Transformer implemented in PyTorch are compared. The results show that BiLSTM achieves the best overall performance with 89% accuracy and a weighted F1-score of 0.89, slightly outperforming the best machine learning model, SVM, which reaches 88.11% accuracy. The findings indicate that while traditional machine learning models remain competitive and computationally efficient, sequence-based deep learning models better capture contextual emotional cues in text.

Related papers

• A Comparative Analysis of Classical Machine Learning and Deep Learning Approaches for Sentiment Classification on IMDb Movie Reviews [0.0]
  This paper presents a comparative study of classical machine learning and deep learning methods for sentiment classification on the IMDb movie reviews dataset.<n>The machine learning pipeline uses TF-IDF features and PyCaret AutoML to evaluate Logistic Regression, Nave Bayes, and Support Vector Machine.<n>The deep learning pipeline implements BiLSTM and BiLSTM with an attention mechanism.
  arXiv  Detail & Related papers  (2026-05-08T14:44:36Z)
• Sentiment Analysis of Indonesian Spotify Reviews Using Machine Learning and BiLSTM [0.0]
  This paper benchmarks machine learning and deep learning approaches for three-class sentiment classification of Indonesian Spotify reviews.<n>BiLSTM is stronger for overall sentiment detection, whereas machine learning with SMOTE provides more balanced three-class performance.
  arXiv  Detail & Related papers  (2026-05-05T07:28:42Z)
• Sentiment Analysis of AI Adoption in Indonesian Higher Education Using Machine Learning and Transformer-Based Models [0.0]
  This study analyzes Indonesian student opinions on the adoption of artificial intelligence in higher education.<n>The dataset consists of 2,295 labeled samples, combining 1,154 student opinions with additional lexical sentiment data.
  arXiv  Detail & Related papers  (2026-04-30T05:25:05Z)
• Benchmarking PyCaret AutoML Against IndoBERT Fine-Tuning for Sentiment Analysis on Indonesian IKN Twitter Data [0.0]
  This paper benchmarks a classical machine learning approach based on PyCaret AutoML against a deep learning approach based on IndoBERT.<n>The dataset contains 1,472 manually labeled samples, consisting of 780 negative and 692 positive comments.
  arXiv  Detail & Related papers  (2026-04-28T09:02:24Z)
• Fine-Grained Emotion Detection on GoEmotions: Experimental Comparison of Classical Machine Learning, BiLSTM, and Transformer Models [4.417564179511245]
  Fine-grained emotion recognition is a challenging multi-label NLP task due to label overlap and class imbalance.<n>We benchmark three modeling families on the GoEmotions dataset: a TF-IDF-based logistic regression system trained with binary relevance, a BiLSTM with attention, and a BERT model fine-tuned for multi-label classification.<n>We observe that logistic regression attains the highest Micro-F1 of 0.51, while BERT achieves the best overall balance surpassing the official paper's reported results.
  arXiv  Detail & Related papers  (2026-01-26T05:29:27Z)
• AI Generated Text Detection [0.0]
  This paper presents an evaluation of AI text detection methods, including both traditional machine learning models and transformer-based architectures.<n>We utilize two datasets, HC3 and DAIGT v2, to build a unified benchmark and apply a topic-based data split to prevent information leakage.<n>Results indicate that contextual modeling is significantly superior to lexical features and highlight the importance of mitigating topic memorization.
  arXiv  Detail & Related papers  (2026-01-07T11:18:10Z)
• Emotion Detection in Speech Using Lightweight and Transformer-Based Models: A Comparative and Ablation Study [0.41292255339309664]
  This paper presents a comparative analysis of lightweight transformer-based models, DistilHuBERT and PaSST.<n>We benchmark their performance against a traditional CNN-LSTM baseline model using MFCC features.<n>DistilHuBERT demonstrates superior accuracy (70.64%) and F1 score (70.36%) while maintaining an exceptionally small model size (0.02 MB)
  arXiv  Detail & Related papers  (2025-11-01T05:01:04Z)
• Logits-Based Finetuning [48.18151583153572]
  We propose a logits-based fine-tuning framework that integrates the strengths of supervised learning and knowledge distillation.<n>Our approach constructs enriched training targets by combining teacher logits with ground truth labels, preserving both correctness and linguistic diversity.
  arXiv  Detail & Related papers  (2025-05-30T10:57:09Z)
• Leveraging Cross-Attention Transformer and Multi-Feature Fusion for Cross-Linguistic Speech Emotion Recognition [60.58049741496505]
  Speech Emotion Recognition (SER) plays a crucial role in enhancing human-computer interaction.<n>We propose a novel approach HuMP-CAT, which combines HuBERT, MFCC, and prosodic characteristics.<n>We show that, by fine-tuning the source model with a small portion of speech from the target datasets, HuMP-CAT achieves an average accuracy of 78.75%.
  arXiv  Detail & Related papers  (2025-01-06T14:31:25Z)
• Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning [55.127202990679976]
  We introduce the MERR dataset, containing 28,618 coarse-grained and 4,487 fine-grained annotated samples across diverse emotional categories. This dataset enables models to learn from varied scenarios and generalize to real-world applications. We propose Emotion-LLaMA, a model that seamlessly integrates audio, visual, and textual inputs through emotion-specific encoders.
  arXiv  Detail & Related papers  (2024-06-17T03:01:22Z)
• Exploring Text-to-Motion Generation with Human Preference [59.28730218998923]
  This paper presents an exploration of preference learning in text-to-motion generation. We find that current improvements in text-to-motion generation still rely on datasets requiring expert labelers with motion capture systems. We show that preference learning has the potential to greatly improve current text-to-motion generative models.
  arXiv  Detail & Related papers  (2024-04-15T04:14:42Z)
• Improving Sampling Methods for Fine-tuning SentenceBERT in Text Streams [49.3179290313959]
  This study explores the efficacy of seven text sampling methods designed to selectively fine-tune language models. We precisely assess the impact of these methods on fine-tuning the SBERT model using four different loss functions. Our findings indicate that Softmax loss and Batch All Triplets loss are particularly effective for text stream classification.
  arXiv  Detail & Related papers  (2024-03-18T23:41:52Z)
• Bag of Tricks for Effective Language Model Pretraining and Downstream Adaptation: A Case Study on GLUE [93.98660272309974]
  This report briefly describes our submission Vega v1 on the General Language Understanding Evaluation leaderboard. GLUE is a collection of nine natural language understanding tasks, including question answering, linguistic acceptability, sentiment analysis, text similarity, paraphrase detection, and natural language inference. With our optimized pretraining and fine-tuning strategies, our 1.3 billion model sets new state-of-the-art on 4/9 tasks, achieving the best average score of 91.3.
  arXiv  Detail & Related papers  (2023-02-18T09:26:35Z)
• DeepEmotex: Classifying Emotion in Text Messages using Deep Transfer Learning [0.0]
  We propose DeepEmotex an effective sequential transfer learning method to detect emotion in text. We conduct an experimental study using both curated Twitter data sets and benchmark data sets. DeepEmotex models achieve over 91% accuracy for multi-class emotion classification on test dataset.
  arXiv  Detail & Related papers  (2022-06-12T03:23:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.