Benchmarking PyCaret AutoML Against IndoBERT Fine-Tuning for Sentiment Analysis on Indonesian IKN Twitter Data

• URL: http://arxiv.org/abs/2604.25392v1
• Date: Tue, 28 Apr 2026 09:02:24 GMT
• Title: Benchmarking PyCaret AutoML Against IndoBERT Fine-Tuning for Sentiment Analysis on Indonesian IKN Twitter Data
• Abstract summary: This paper benchmarks a classical machine learning approach based on PyCaret AutoML against a deep learning approach based on IndoBERT.<n>The dataset contains 1,472 manually labeled samples, consisting of 780 negative and 692 positive comments.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper benchmarks a classical machine learning approach based on PyCaret AutoML against a deep learning approach based on IndoBERT fine-tuning for binary sentiment analysis of Indonesian-language Twitter comments related to Ibu Kota Nusantara (IKN). The dataset contains 1,472 manually labeled samples, consisting of 780 negative and 692 positive comments. In the machine learning setting, Logistic Regression, Naive Bayes, and Support Vector Machine were evaluated using 10-fold cross-validation, with Logistic Regression achieving the best performance among the classical models at 77.57% accuracy and 77.17% F1-score. In the deep learning setting, the indobenchmark/indobert-base-p1 model was fine-tuned for five epochs and achieved 89.59% test accuracy and 89.37% F1-score. The results show that IndoBERT substantially outperforms the machine learning baselines, highlighting the effectiveness of Transformer-based contextual representations for informal Indonesian social media text.

Related papers

• A Controlled Synthetic Benchmark for Educational Aspect-Based Sentiment Analysis [47.027290803102666]
  This study introduces a controlled benchmark for educational ABSA built from 10,000 synthetic course reviews.<n>The strongest untuned model, BERT, reaches a held-out detection micro-F1 of 0.2760.<n>A conservative external evaluation on 2,829 mapped student-feedback reviews from Herath et al. yields a micro-F1 of 0.4593 for BERT on a 9-aspect overlap.
  arXiv  Detail & Related papers  (2026-05-25T07:05:21Z)
• A Comparative Study of PyCaret AutoML and CNN-BiLSTM for Binary Hate Speech Detection in Indonesian Twitter [0.0]
  This paper compares a PyCaret AutoML branch and a CNN-BiLSTM branch for binary hate speech detection on Indonesian Twitter using the HS label from the corpus of Ibrohim and Budi.<n>The conventional branch uses TF-IDF with a lexicon-based abusive-word count, whereas the neural branch learns dense token representations and captures both local phrase patterns and bidirectional context.<n>On the held-out split, CNN-BiLSTM achieves the best result with 83.8% accuracy, 79.8% precision, 82.7% recall, and 81.2% F1-score.
  arXiv  Detail & Related papers  (2026-05-06T13:20:51Z)
• Benchmarking Logistic Regression, SVM, Naive Bayes, and IndoBERT Fine-Tuning for Sentiment Analysis on Indonesian Product Reviews [0.0]
  This paper benchmarks traditional Machine Learning approaches against a Transformer-based Deep Learning model for a three-class sentiment analysis task.<n>We fine-tuned the IndoBERT model for contextual sequence classification.<n>We demonstrate the practical viability of our pipeline by deploying the final sentiment classification model as an interactive Gradio web application.
  arXiv  Detail & Related papers  (2026-05-05T07:20:42Z)
• Sentiment Analysis of AI Adoption in Indonesian Higher Education Using Machine Learning and Transformer-Based Models [0.0]
  This study analyzes Indonesian student opinions on the adoption of artificial intelligence in higher education.<n>The dataset consists of 2,295 labeled samples, combining 1,154 student opinions with additional lexical sentiment data.
  arXiv  Detail & Related papers  (2026-04-30T05:25:05Z)
• Benchmarking PyCaret AutoML Against BiLSTM for Fine-Grained Emotion Classification: A Comparative Study on 20-Class Emotion Detection [0.0]
  This study benchmarks classical machine learning and deep learning approaches for 20-class emotion classification.<n>The results show that BiLSTM achieves the best overall performance with 89% accuracy and a weighted F1-score of 0.89.
  arXiv  Detail & Related papers  (2026-04-29T05:31:45Z)
• Benchmarking Logistic Regression, SVM, and LightGBM Against BiLSTM with Attention for Sentiment Analysis on Indonesian Product Reviews [0.0]
  This paper compares a Machine Learning (ML) approach via the PyCaret AutoML framework against a Deep Learning (DL) approach based on a Bidirectional Long Short-Term Memory (BiLSTM) architecture with an Attention mechanism for binary sentiment classification on Indonesian product reviews.<n> Logistic Regression achieved the best ML performance with an accuracy of 97.26% and an F1-score of 97.26%.
  arXiv  Detail & Related papers  (2026-04-28T10:00:42Z)
• Preference Optimization for Review Question Generation Improves Writing Quality [29.997337923485933]
  We develop a novel reward model built from a frozen autoregressive LLM with trainable multi-head transformers.<n>We train a question-generation model aligned with human standards of effort, evidence, and grounding.<n>Compared to the Qwen3-32B base model, IntelliAsk shows measurable gains across diverse benchmarks.
  arXiv  Detail & Related papers  (2026-01-23T18:58:22Z)
• Fantastic Bugs and Where to Find Them in AI Benchmarks [28.604919035475188]
  We introduce a framework for systematic benchmark revision that leverages statistical analysis of response patterns to flag potentially invalid questions.<n>Our approach builds on a core assumption commonly used in AI evaluations that the mean score sufficiently summarizes model performance.<n>Across nine widely used benchmarks, our method guides expert review to identify problematic questions with up to 84% precision.
  arXiv  Detail & Related papers  (2025-11-20T22:49:21Z)
• HausaMovieReview: A Benchmark Dataset for Sentiment Analysis in Low-Resource African Language [1.3465808629549525]
  This paper introduces a novel benchmark dataset comprising 5,000 YouTube comments in Hausa and code-switched English.<n>We use this dataset to conduct a comparative analysis of classical models and fine-tuned transformer models.<n>Our results reveal a key finding: the Decision Tree classifier, with an accuracy and F1-score 89.72% and 89.60% respectively, significantly outperformed the deep learning models.
  arXiv  Detail & Related papers  (2025-09-17T22:57:21Z)
• Logits-Based Finetuning [48.18151583153572]
  We propose a logits-based fine-tuning framework that integrates the strengths of supervised learning and knowledge distillation.<n>Our approach constructs enriched training targets by combining teacher logits with ground truth labels, preserving both correctness and linguistic diversity.
  arXiv  Detail & Related papers  (2025-05-30T10:57:09Z)
• RideKE: Leveraging Low-Resource, User-Generated Twitter Content for Sentiment and Emotion Detection in Kenyan Code-Switched Dataset [4.142287865325998]
  We analyze Kenyan code-switched data and evaluate four state-of-the-art (SOTA) transformer-based pretrained models for sentiment and emotion classification.<n>For sentiment analysis, XLM-R supervised model achieves the highest accuracy (69.2%) and F1 score (66.1%), XLM-R semi-supervised (67.2% accuracy, 64.1% F1 score)<n>In emotion analysis, DistilBERT supervised leads in accuracy (59.8%) and F1 score (31%), mBERT semi-supervised (accuracy (59% and F1 score 26.5
  arXiv  Detail & Related papers  (2025-02-10T06:18:07Z)
• Implicit Sentiment Analysis Based on Chain of Thought Prompting [1.4582633500696451]
  This paper introduces a Sentiment Analysis of Thinking (SAoT) framework. The framework first analyzes the implicit aspects and opinions in the text using common sense and thinking chain capabilities. The model is evaluated on the SemEval 2014 dataset, consisting of 1120 restaurant reviews and 638 laptop reviews.
  arXiv  Detail & Related papers  (2024-08-22T06:55:29Z)
• VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Use [49.574651930395305]
  VisIT-Bench is a benchmark for evaluation of instruction-following vision-language models. Our dataset comprises 592 test queries, each with a human-authored instruction-conditioned caption. We quantify quality gaps between models and references using both human and automatic evaluations.
  arXiv  Detail & Related papers  (2023-08-12T15:27:51Z)
• BERT Fine-Tuning for Sentiment Analysis on Indonesian Mobile Apps Reviews [1.5749416770494706]
  This study examines the effectiveness of fine-tuning BERT for sentiment analysis using two different pre-trained models. The dataset used is Indonesian user reviews of the ten best apps in 2020 in Google Play sites. Two training data labeling approaches were also tested to determine the effectiveness of the model, which is score-based and lexicon-based.
  arXiv  Detail & Related papers  (2021-07-14T16:00:15Z)
• DeBERTa: Decoding-enhanced BERT with Disentangled Attention [119.77305080520718]
  We propose a new model architecture DeBERTa that improves the BERT and RoBERTa models using two novel techniques. We show that these techniques significantly improve the efficiency of model pre-training and the performance of both natural language understanding (NLU) and natural langauge generation (NLG) downstream tasks.
  arXiv  Detail & Related papers  (2020-06-05T19:54:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.